-- Logs begin at Mon 2022-10-10 11:19:24 UTC, end at Mon 2022-10-10 12:16:57 UTC. -- Oct 10 11:44:40.531013 dv-cinder-850618-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 10 11:44:40.539627 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 10 11:44:40 2022] *** Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: nodename: dv-cinder-850618-4-iscsi Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: machine: x86_64 Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: clock source: unix Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: pcre jit disabled Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: detected number of CPU cores: 4 Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: current working directory: / Oct 10 11:44:40.541616 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: detected binary path: /usr/bin/uwsgi-core Oct 10 11:44:40.542260 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: your processes number limit is 47680 Oct 10 11:44:40.542260 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: your memory page size is 4096 bytes Oct 10 11:44:40.542260 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: detected max file descriptor number: 2048 Oct 10 11:44:40.542260 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: lock engine: pthread robust mutexes Oct 10 11:44:40.542260 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: thunder lock: enabled Oct 10 11:44:40.542260 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 10 11:44:40.542260 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 10 11:44:40.581422 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: Python main interpreter initialized at 0x556f1673d9c0 Oct 10 11:44:40.581422 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: python threads support enabled Oct 10 11:44:40.581422 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: your server socket listen backlog is limited to 100 connections Oct 10 11:44:40.581422 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: your mercy for graceful operations on workers is 90 seconds Oct 10 11:44:40.581768 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: mapped 403077 bytes (393 KB) for 2 cores Oct 10 11:44:40.581857 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: *** Operational MODE: preforking *** Oct 10 11:44:40.582100 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: *** uWSGI is running in multiple interpreter mode *** Oct 10 11:44:40.582100 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: spawned uWSGI master process (pid: 81445) Oct 10 11:44:40.582297 dv-cinder-850618-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 10 11:44:40.586929 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: spawned uWSGI worker 1 (pid: 81457, cores: 1) Oct 10 11:44:40.586929 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: spawned uWSGI worker 2 (pid: 81458, cores: 1) Oct 10 11:44:40.586929 dv-cinder-850618-4-iscsi devstack@keystone.service[81445]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 10 11:44:44.847033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 11:44:44.847033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: from cryptography.utils import int_from_bytes Oct 10 11:44:44.847783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 11:44:44.847783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: from cryptography.utils import int_from_bytes Oct 10 11:44:44.847783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 11:44:44.847783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: from cryptography.utils import int_from_bytes Oct 10 11:44:44.848297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 11:44:44.848297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: from cryptography.utils import int_from_bytes Oct 10 11:44:44.897335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 10 11:44:44.897502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 10 11:44:44.897587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 10 11:44:44.897678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 10 11:44:44.897751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] command line args: [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 10 11:44:44.897827 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] command line args: [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 10 11:44:44.897946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 10 11:44:44.898024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 10 11:44:44.898104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ================================================================================ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 10 11:44:44.898181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ================================================================================ {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 10 11:44:44.898355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] admin_token = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.898441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] admin_token = **** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.898600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_dir = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.898689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] config_dir = [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.898785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.898871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.898984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_source = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.899057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] config_source = [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.899157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] debug = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.899248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] debug = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.899352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.899656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.899782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.899890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.899986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900485 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] list_limit = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_config_append = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] log_config_append = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.900977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_dir = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] log_dir = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] log_file = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.901996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.902938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.903963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.904078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.904160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.904317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.904442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.904584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.904661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] publish_errors = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] publish_errors = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.905966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.906081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.906178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.906309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.906393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.906536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] standard_threads = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.906654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] standard_threads = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.906761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.906878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] use_journal = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_journal = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] use_json = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_json = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907727 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] use_stderr = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_stderr = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.907878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] use_syslog = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.908001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_syslog = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.908086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.908158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:44:44.908280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.908358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.908510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.908586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.908762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.908840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.908997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.909074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.909240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.909322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.909473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.909556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.909714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.909792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.909945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.910027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.910186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.910263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.910426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.910508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.910667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.910744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.910918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.911947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.912098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.912177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.912348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.912427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.912595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.912671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.912833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.912923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.913074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.913153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.913300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.913377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.913551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.913635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.913797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.913873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.914965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.915979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.916163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.916245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.916408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.916494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.916641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.916724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.916882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.916958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.917106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.917194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.917338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.917457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.917559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.917636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.917782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.917872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.918926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.919926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.920986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.921983 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.922098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.922177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.922326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.922402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.922556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.922635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.922780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.922866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923056 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.923964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.924051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.924228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.924306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.924458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.924575 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.924699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.924776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.924935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.925013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.925165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.925262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.925389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.925466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.925617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.925788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.925933 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926356 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.926989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.927978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.928131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.928207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.928386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.928465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.928618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.928695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.928859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.928937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.929091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.929169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.929320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.929396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.929554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.929631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.929777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.929858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.930989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.931155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.931245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.931379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.931456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.932135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.932227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.932379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.932465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.932623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.932701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.932853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.932930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.933103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.933187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.933354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.933432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.933586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.933692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.933809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] auth.external = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.933888 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.external = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.934033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.934118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.934319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.934399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.934542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.934621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.934768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.password = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.934858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] auth.password = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.token = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] auth.token = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.935930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] credential.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.936947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.937102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.937180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.937344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.937422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.937586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.937673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.937822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.937900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.938059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.938149 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.938288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.938375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.938529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.938607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.938774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.938853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.939029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.939123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.939277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.939355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.939551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.939629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.939790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.939876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.940045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.940122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.940280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.940358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.940776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.940870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] federation.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.941944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942417 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.942995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.943152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.943241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.943381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.943473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.943619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.943700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.943848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.943926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.944090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.944168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.944324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.944401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.944551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.944631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.944781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.944859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.945960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.946041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.946184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.946261 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.946406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.946486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.946629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.946707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.946865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.947003 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.947128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.947210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.947911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.948980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.949132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.949212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.949355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.949433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.949584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.949662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.949807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.949893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.950993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.951074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.951236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.951316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.951477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.951563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.951708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.951786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.951950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952417 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.952950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.953103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.953193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.953333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.953411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.953568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.953646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.953799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.953885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.954137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.954215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.954385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.954461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.954630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.954715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.954859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.954986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.955109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.955187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.955352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.955443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.955596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.955676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.955827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.955952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956146 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.956998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.957077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.957225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.957302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.957468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.957554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.957710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.957789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.957951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.958970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.959117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.959195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.959342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.959419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.959583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.959670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.959824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.959902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.960065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.960144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.960306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.960392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.960545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.960623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.960769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.960846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961780 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.961922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.962000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.962158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.962236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.962403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.962490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.962632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.962710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.962937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963621 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.963936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.964080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.964159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.964322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.964399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.964556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.964642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.964782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.964860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] resource.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.965958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.966048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.966200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.966278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.966441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.966519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.966670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.966755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.966922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] role.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] role.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.967927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.968071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.968151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.968311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.968388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.968552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.968631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.968793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.968872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.969994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.970072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.970219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.970298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.970459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.970536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.970697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.970775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.970960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.971044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.971205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.971541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.971622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.971693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.971808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.971897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.972999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.973085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.973245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.973324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.973490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.973567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.973721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.973808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.973945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.974987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.975141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.975231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.975372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.975450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.975600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.975677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.975822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.975908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] token.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976356 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.976998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.977075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.977239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.977324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.977471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.977550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.977707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.977785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.977942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.978028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.978184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.978261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.978415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.978520 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.978672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.978760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.978924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.979982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.980156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.980236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.980411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.980490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.connection = **** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.980651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.980732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.980893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.980970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.981123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.981201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.981359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.981438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.981587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.981664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.981814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.981891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.982992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.983074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.983223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.983301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.983465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.983545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.983692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.983769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.983927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.984004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.984151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.984230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.984383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.984460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.984613 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.984691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:44:44.984772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81457) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 10 11:44:44.984845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 10 11:44:45.014525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.014752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.014835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.015002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.015090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.015259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.023307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.023498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.052671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.052896 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.053757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.053990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.106461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.106604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.155273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.155505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.155591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.155812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.311519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.311728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.311814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.311889 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.311962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.312076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.312169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.312294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.312376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.312508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.312590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.312722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.335886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.335993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.336968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.337050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.337175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.337256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.348790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.348965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.362881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.363051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.363147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.363266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.363348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81457) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.363477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:44:45.380463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.380671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 11:44:45.380825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.381010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.381091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 11:44:45.381207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 11:44:45.381356 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.381478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.381560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 11:44:45.381736 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.381816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.381991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.382079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.382246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.382326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.382540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.392412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.392677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.392773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.392942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.393025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.393206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.393286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.393484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.400193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.401048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.403699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.403978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.404156 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.404305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.404446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.405897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.406039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.406125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.406304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.406387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.406586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.428980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.429327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.430434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.430717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.430799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.431084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.435458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.435736 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.435817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.436011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.436092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.436323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.442611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.442876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.442979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.443225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.449086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.449377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.449470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.449664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.449747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.449951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.450046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.450228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.450310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.450499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.450581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.450766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.450850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.451055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.451150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.451328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.467239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.467350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.467514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.467598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.471357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.471443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.474580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.474664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.474875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.474977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.479653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.479761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.482773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.482858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.486003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.486085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.486303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.486387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.486586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.486668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.486870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.487031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.487185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.487267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.487477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.487560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.503759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.503852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.504049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.504131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.504319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.504400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.504581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.504665 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.511444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.511528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.512987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.513145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.514510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.514606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.514783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.514874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.517456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.517563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.517741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.517834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.518021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.518111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.518301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.518393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.518692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.518795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.530357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.530482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.530641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.530731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.530939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.531040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.531219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.531304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.531594 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.531674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.531876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.531965 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.532149 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.532230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.548068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.548163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.550963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.551058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.553812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.553903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.555273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.555441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.557239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.557384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.557547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.557667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.557897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.558061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.564250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.564426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.565951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.566132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.568861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.569028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.569259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.569418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.569542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.569701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.569820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.569975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.578216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.578343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.578500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.578630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.578786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.578931 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.585654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.585789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81457) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:44:45.585936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.586069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.586213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.586346 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.586484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.586617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.586755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.586911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.587050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.587178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.587332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.587463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.587611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.587738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.587912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.588062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.588189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.588344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.588466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.588613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.588724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.588870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.589002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.589140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.589270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.589487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81457) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:44:45.623761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556f1673d9c0 pid: 81457 (default app) Oct 10 11:44:45.625089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556f1673d9c0 pid: 81458 (default app) Oct 10 11:44:45.626932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ec258-b1e2-4e41-88b2-14eff5f48275 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:45.627220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ec258-b1e2-4e41-88b2-14eff5f48275 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:45.627489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ec258-b1e2-4e41-88b2-14eff5f48275 None None] PATH_INFO: `/v3` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:45.631629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Mon Oct 10 11:44:45 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:44:52.369307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9b820-1eea-467f-94a7-690e72c1d2de None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:52.369824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9b820-1eea-467f-94a7-690e72c1d2de None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:52.369908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9b820-1eea-467f-94a7-690e72c1d2de None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:52.374151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:44:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:44:52.379478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:52.379943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:52.380078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:52.450622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: INFO dbcounter [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] Registered counter for database keystone Oct 10 11:44:52.469834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG oslo_db.sqlalchemy.engines [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81457) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 10 11:44:52.475994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writer thread running {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 10 11:44:52.652901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG passlib.handlers.bcrypt [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81457) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 10 11:44:52.653204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG passlib.handlers.bcrypt [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81457) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 10 11:44:52.710145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:44:52.711766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:44:52.838287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:44:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 11:44:53.292503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9b820-1eea-467f-94a7-690e72c1d2de None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:53.362205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: INFO dbcounter [None req-33e9b820-1eea-467f-94a7-690e72c1d2de None None] Registered counter for database keystone Oct 10 11:44:53.383231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG oslo_db.sqlalchemy.engines [None req-33e9b820-1eea-467f-94a7-690e72c1d2de None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81458) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 10 11:44:53.390037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writer thread running {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 10 11:44:53.507477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9b820-1eea-467f-94a7-690e72c1d2de None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:53.512297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7ae24-d12a-4075-8990-b0886eb1a032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:53.512928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7ae24-d12a-4075-8990-b0886eb1a032 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:53.513213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7ae24-d12a-4075-8990-b0886eb1a032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:53.513489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7ae24-d12a-4075-8990-b0886eb1a032 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:53.521659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7ae24-d12a-4075-8990-b0886eb1a032 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:53.573952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4db7ae24-d12a-4075-8990-b0886eb1a032 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 10 11:44:53.574298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7ae24-d12a-4075-8990-b0886eb1a032 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:53.583552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Mon Oct 10 11:44:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 11:44:53.590077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:53.606392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6827e6-d819-4b06-999c-d892410eaa43 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:53.610704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf305f-c3c2-4dae-b003-a6e8ed35c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:53.611826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf305f-c3c2-4dae-b003-a6e8ed35c629 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:53.612107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf305f-c3c2-4dae-b003-a6e8ed35c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:53.612376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf305f-c3c2-4dae-b003-a6e8ed35c629 admin admin] PATH_INFO: `/v3/projects/992fcf5a85d6497db94cb36327aec1f0` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:53.613927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf305f-c3c2-4dae-b003-a6e8ed35c629 admin admin] RBAC: Authorizing `identity:get_project(project_id=992fcf5a85d6497db94cb36327aec1f0)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:53.667941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5bbf305f-c3c2-4dae-b003-a6e8ed35c629 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 10 11:44:53.668279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf305f-c3c2-4dae-b003-a6e8ed35c629 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:53.672777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Mon Oct 10 11:44:53 2022] GET /identity/v3/projects/992fcf5a85d6497db94cb36327aec1f0 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:44:55.772683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b211d7-a947-44a0-b5a9-b9494d9718f7 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:55.773179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b211d7-a947-44a0-b5a9-b9494d9718f7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:55.773179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b211d7-a947-44a0-b5a9-b9494d9718f7 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:55.777318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:44:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:44:55.782834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bbcb6-566f-4fc3-abd6-4aa8223f07d2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:55.783260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bbcb6-566f-4fc3-abd6-4aa8223f07d2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:55.783435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bbcb6-566f-4fc3-abd6-4aa8223f07d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:55.804065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b44bbcb6-566f-4fc3-abd6-4aa8223f07d2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:44:55.805572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b44bbcb6-566f-4fc3-abd6-4aa8223f07d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:44:55.813942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:44:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 11:44:56.079218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b211d7-a947-44a0-b5a9-b9494d9718f7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:56.091810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b211d7-a947-44a0-b5a9-b9494d9718f7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:56.096051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf3978-6dc3-4cc7-b59c-af93e7133725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:56.096649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf3978-6dc3-4cc7-b59c-af93e7133725 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:56.096932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf3978-6dc3-4cc7-b59c-af93e7133725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:56.097206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf3978-6dc3-4cc7-b59c-af93e7133725 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:56.106748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf3978-6dc3-4cc7-b59c-af93e7133725 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:56.109493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf3978-6dc3-4cc7-b59c-af93e7133725 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:56.118183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Mon Oct 10 11:44:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 11:44:56.124234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bbcb6-566f-4fc3-abd6-4aa8223f07d2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:56.133214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bbcb6-566f-4fc3-abd6-4aa8223f07d2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:56.137538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da14437-7262-40e4-805a-e34063728f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:56.138160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da14437-7262-40e4-805a-e34063728f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:56.138442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da14437-7262-40e4-805a-e34063728f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:56.138712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da14437-7262-40e4-805a-e34063728f88 admin admin] PATH_INFO: `/v3/users/10d738bcb5484a248894a50e95518057` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:56.140036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da14437-7262-40e4-805a-e34063728f88 admin admin] RBAC: Authorizing `identity:get_user(user_id=10d738bcb5484a248894a50e95518057)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:56.141846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da14437-7262-40e4-805a-e34063728f88 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:56.143385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Mon Oct 10 11:44:56 2022] GET /identity/v3/users/10d738bcb5484a248894a50e95518057 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:44:58.349431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cdab5-b577-4951-8786-2d91bdd03db6 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:58.349980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cdab5-b577-4951-8786-2d91bdd03db6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:58.349980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cdab5-b577-4951-8786-2d91bdd03db6 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:58.351334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:44:58.356657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf5911-5c8a-4719-8dcd-861d75335158 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:58.357051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf5911-5c8a-4719-8dcd-861d75335158 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:58.357212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf5911-5c8a-4719-8dcd-861d75335158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:58.365707 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aea53-28cb-491f-ae01-582f36f1db0f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:58.365988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aea53-28cb-491f-ae01-582f36f1db0f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:58.366263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aea53-28cb-491f-ae01-582f36f1db0f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:58.367556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:44:58.372955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:58.373245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:58.373525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:58.375863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bfcf5911-5c8a-4719-8dcd-861d75335158 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:44:58.377034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bfcf5911-5c8a-4719-8dcd-861d75335158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:44:58.415724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG passlib.handlers.bcrypt [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81458) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 10 11:44:58.416123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG passlib.handlers.bcrypt [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81458) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 10 11:44:58.457539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:44:58.466256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:44:58.467887 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:44:58.567034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:44:58.723496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf5911-5c8a-4719-8dcd-861d75335158 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:58.749004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:44:58.827524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf5911-5c8a-4719-8dcd-861d75335158 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:58.834420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:58.861221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d121d0c-4be2-4f53-a9f7-da0647eadc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:58.861849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d121d0c-4be2-4f53-a9f7-da0647eadc04 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:58.862124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d121d0c-4be2-4f53-a9f7-da0647eadc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:58.862400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d121d0c-4be2-4f53-a9f7-da0647eadc04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:58.873585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d121d0c-4be2-4f53-a9f7-da0647eadc04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-9d121d0c-4be2-4f53-a9f7-da0647eadc04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:44:58.882198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:44:58.883297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:44:58.884178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:44:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:44:58.889164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d121d0c-4be2-4f53-a9f7-da0647eadc04 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:58.935074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afb8f1-fb05-4ce7-b89c-b1162cffb5b1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:58.938047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:44:58.969082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:44:58.973101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135e5c5-68bd-4e9a-80e0-a342581d4d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:58.974045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135e5c5-68bd-4e9a-80e0-a342581d4d02 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:58.974287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135e5c5-68bd-4e9a-80e0-a342581d4d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:58.974563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135e5c5-68bd-4e9a-80e0-a342581d4d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:58.987614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d121d0c-4be2-4f53-a9f7-da0647eadc04 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:59.020984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:59.021590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:59.021867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:59.022134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:59.022795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:44:59.023125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:59.024516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:59.036750 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:44:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:44:59.041667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135e5c5-68bd-4e9a-80e0-a342581d4d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:59.043115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:59.043233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135e5c5-68bd-4e9a-80e0-a342581d4d02 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:59.139975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:44:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:44:59.140523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0a4b9-3d4a-4871-8190-31f171460691 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:59.145324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135e5c5-68bd-4e9a-80e0-a342581d4d02 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:59.148069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:44:59.174114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188f3ed-2d91-467c-a7a9-d372c4434c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:59.174741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188f3ed-2d91-467c-a7a9-d372c4434c3b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:59.175060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188f3ed-2d91-467c-a7a9-d372c4434c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:59.175338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188f3ed-2d91-467c-a7a9-d372c4434c3b None admin] PATH_INFO: `/v3/users/10d738bcb5484a248894a50e95518057` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:59.176493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188f3ed-2d91-467c-a7a9-d372c4434c3b None admin] RBAC: Authorizing `identity:get_user(user_id=10d738bcb5484a248894a50e95518057)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:59.177945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188f3ed-2d91-467c-a7a9-d372c4434c3b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:59.179544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:44:59 2022] GET /identity/v3/users/10d738bcb5484a248894a50e95518057 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:44:59.184832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188f3ed-2d91-467c-a7a9-d372c4434c3b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:59.199097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:44:59.244007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135e5c5-68bd-4e9a-80e0-a342581d4d02 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:59.277707 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:59.278566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:59.278849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:59.279151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:59.284517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188f3ed-2d91-467c-a7a9-d372c4434c3b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:59.298026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:59.299534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:59.317606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8961269-26ac-413b-8a40-8b96ba875696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:44:59.317806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:44:59.318636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:44:59.319672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:44:59.319672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:44:59.319672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:44:59.319842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8961269-26ac-413b-8a40-8b96ba875696 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:59.319842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8961269-26ac-413b-8a40-8b96ba875696 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:59.319842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8961269-26ac-413b-8a40-8b96ba875696 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:59.320020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:44:59.320304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8961269-26ac-413b-8a40-8b96ba875696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:59.321804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8961269-26ac-413b-8a40-8b96ba875696 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:59.323385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:44:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:44:59.325085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:59.328865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8961269-26ac-413b-8a40-8b96ba875696 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:44:59.382021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:44:59.409914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:44:59.428891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3aa8c-1ddc-4b1a-ab78-b10bd49203c2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:59.435746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8961269-26ac-413b-8a40-8b96ba875696 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:44:59.463499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fa038-a3bd-4ba8-8f3f-eed620e20c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:59.464105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fa038-a3bd-4ba8-8f3f-eed620e20c7a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:59.464391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fa038-a3bd-4ba8-8f3f-eed620e20c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:59.464662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fa038-a3bd-4ba8-8f3f-eed620e20c7a None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:59.465303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fa038-a3bd-4ba8-8f3f-eed620e20c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:44:59.465602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fa038-a3bd-4ba8-8f3f-eed620e20c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:59.467010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fa038-a3bd-4ba8-8f3f-eed620e20c7a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:59.469637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25e005-6035-43b7-9425-a7c5c1b0df19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:44:59.470238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25e005-6035-43b7-9425-a7c5c1b0df19 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:44:59.470519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25e005-6035-43b7-9425-a7c5c1b0df19 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:44:59.470801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25e005-6035-43b7-9425-a7c5c1b0df19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:44:59.471552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25e005-6035-43b7-9425-a7c5c1b0df19 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.domain.id=default, user.id=10d738bcb5484a248894a50e95518057) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:44:59.471856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25e005-6035-43b7-9425-a7c5c1b0df19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:44:59.473283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25e005-6035-43b7-9425-a7c5c1b0df19 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:44:59.479834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Mon Oct 10 11:44:59 2022] GET /identity/v3/role_assignments?user.id=10d738bcb5484a248894a50e95518057&scope.domain.id=default&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:44:59.483984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:01.954642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253085-e17e-4f4c-a56e-e519609f1bb5 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:01.955203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253085-e17e-4f4c-a56e-e519609f1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:01.955203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253085-e17e-4f4c-a56e-e519609f1bb5 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:01.956859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:01.963040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ffe8-a8c4-4596-b7d7-ba486dfeb23b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:01.963442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ffe8-a8c4-4596-b7d7-ba486dfeb23b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:01.963611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ffe8-a8c4-4596-b7d7-ba486dfeb23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:01.983120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-df42ffe8-a8c4-4596-b7d7-ba486dfeb23b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:01.984342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df42ffe8-a8c4-4596-b7d7-ba486dfeb23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:02.058806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:02.333306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21173c2a-adc5-408b-9e25-1be64a400c1a None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:02.333678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21173c2a-adc5-408b-9e25-1be64a400c1a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:02.333874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21173c2a-adc5-408b-9e25-1be64a400c1a None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:02.335658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:02.341736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037b571-8412-4a8b-8d06-a1187c8f93ce None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:02.342038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037b571-8412-4a8b-8d06-a1187c8f93ce None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:02.342306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037b571-8412-4a8b-8d06-a1187c8f93ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:02.367923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b037b571-8412-4a8b-8d06-a1187c8f93ce None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:02.369673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b037b571-8412-4a8b-8d06-a1187c8f93ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:02.388544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21173c2a-adc5-408b-9e25-1be64a400c1a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:02.459561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:02.517281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 12/25] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:02.519139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca71ef-e101-4a20-b4cc-1a0665d93b83 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:02.519652 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca71ef-e101-4a20-b4cc-1a0665d93b83 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:02.520017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca71ef-e101-4a20-b4cc-1a0665d93b83 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:02.525269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 13/26] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:02.526796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beddfbd-0bf5-4aed-9388-9aaf6b4748fa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:02.527304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beddfbd-0bf5-4aed-9388-9aaf6b4748fa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:02.527748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beddfbd-0bf5-4aed-9388-9aaf6b4748fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:02.541969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21173c2a-adc5-408b-9e25-1be64a400c1a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:02.556904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0beddfbd-0bf5-4aed-9388-9aaf6b4748fa None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:02.558446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0beddfbd-0bf5-4aed-9388-9aaf6b4748fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:02.582347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024cd9f-561c-4d54-a745-2d3f7a7f2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:02.583241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024cd9f-561c-4d54-a745-2d3f7a7f2b76 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:02.583775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024cd9f-561c-4d54-a745-2d3f7a7f2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:02.584068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024cd9f-561c-4d54-a745-2d3f7a7f2b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:02.597484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024cd9f-561c-4d54-a745-2d3f7a7f2b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:02.599376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024cd9f-561c-4d54-a745-2d3f7a7f2b76 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:02.610791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-8024cd9f-561c-4d54-a745-2d3f7a7f2b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:45:02.621662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:02.622521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:45:02.623369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:45:02.623369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:45:02.623369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:02.630088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 14/27] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:02.631720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22703da4-7afe-4f29-8fa4-7bf20034375a None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:02.632297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22703da4-7afe-4f29-8fa4-7bf20034375a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:02.632760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22703da4-7afe-4f29-8fa4-7bf20034375a None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:02.635212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 15/28] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:02.636286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22703da4-7afe-4f29-8fa4-7bf20034375a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:02.717713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 14/29] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:02.719010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d851cb5-974f-4d6f-a935-1b679ca2df2a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:02.719302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d851cb5-974f-4d6f-a935-1b679ca2df2a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:02.719579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d851cb5-974f-4d6f-a935-1b679ca2df2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:02.760839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2d851cb5-974f-4d6f-a935-1b679ca2df2a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:02.762049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2d851cb5-974f-4d6f-a935-1b679ca2df2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:02.768525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22703da4-7afe-4f29-8fa4-7bf20034375a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:02.853031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:02.853563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:02.853843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:02.854113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:02.854760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:02.855092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:02.856480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:02.892509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 16/30] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:02.893339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:02.912344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 15/31] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:02.913179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d851cb5-974f-4d6f-a935-1b679ca2df2a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:02.975253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:03.008096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:03.051861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d851cb5-974f-4d6f-a935-1b679ca2df2a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:03.091146 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2e867-2629-4ac0-a586-d02812c6721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:03.092123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2e867-2629-4ac0-a586-d02812c6721f None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:03.092605 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2e867-2629-4ac0-a586-d02812c6721f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:03.093064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2e867-2629-4ac0-a586-d02812c6721f None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:03.094138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2e867-2629-4ac0-a586-d02812c6721f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:03.096025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2e867-2629-4ac0-a586-d02812c6721f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:03.117816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6f5ab-0294-4721-8a23-48a62c121446 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:03.179674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e32e7-9b22-4024-a9b6-60ded50595ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:03.180301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e32e7-9b22-4024-a9b6-60ded50595ca None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:03.180586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e32e7-9b22-4024-a9b6-60ded50595ca None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:03.180891 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e32e7-9b22-4024-a9b6-60ded50595ca None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:03.181573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e32e7-9b22-4024-a9b6-60ded50595ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:03.183008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e32e7-9b22-4024-a9b6-60ded50595ca None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:03.296694 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 16/32] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:45:02 2022] POST /identity/v3/projects => generated 301 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:03.297709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2e867-2629-4ac0-a586-d02812c6721f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:03.354079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 17/33] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Oct 10 11:45:02 2022] POST /identity/v3/roles => generated 219 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:03.355145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e32e7-9b22-4024-a9b6-60ded50595ca None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:03.500030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:03.518001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2e867-2629-4ac0-a586-d02812c6721f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:03.557607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e32e7-9b22-4024-a9b6-60ded50595ca None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:03.589516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d77759-8a97-491c-b9de-b85e9f83dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:03.590164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d77759-8a97-491c-b9de-b85e9f83dec3 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:03.590451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d77759-8a97-491c-b9de-b85e9f83dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:03.590731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d77759-8a97-491c-b9de-b85e9f83dec3 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:03.591469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d77759-8a97-491c-b9de-b85e9f83dec3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:03.593309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d77759-8a97-491c-b9de-b85e9f83dec3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:03.603775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:03.628697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 17/34] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Oct 10 11:45:03 2022] POST /identity/v3/roles => generated 225 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:03.632264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a192bd-95ad-4808-acc0-02d29cf6f7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:03.633036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a192bd-95ad-4808-acc0-02d29cf6f7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:03.633330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a192bd-95ad-4808-acc0-02d29cf6f7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:03.633659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a192bd-95ad-4808-acc0-02d29cf6f7f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:03.648769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a192bd-95ad-4808-acc0-02d29cf6f7f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-d2a192bd-95ad-4808-acc0-02d29cf6f7f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:03.655221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:03.656086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:03.657850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 18/35] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:03.662853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d77759-8a97-491c-b9de-b85e9f83dec3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:03.764305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439be5e7-42e5-4846-9724-556072906367 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:03.764625 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439be5e7-42e5-4846-9724-556072906367 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:03.764867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439be5e7-42e5-4846-9724-556072906367 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:03.766503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 19/36] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:03.771928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c18b-2cff-4d0a-bbe6-b92a270a09c3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:03.772222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c18b-2cff-4d0a-bbe6-b92a270a09c3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:03.772503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c18b-2cff-4d0a-bbe6-b92a270a09c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:03.789756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d77759-8a97-491c-b9de-b85e9f83dec3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:03.798120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a4c1c18b-2cff-4d0a-bbe6-b92a270a09c3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:03.799552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a4c1c18b-2cff-4d0a-bbe6-b92a270a09c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:03.829313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:03.830370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:03.830726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:03.831038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:03.831779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:03.832082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:03.833688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:03.847341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 18/37] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:03.853851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:03.867730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:03.979797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a1c2-4b87-4d15-a5ff-e022bb6e0124 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:04.017094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 20/38] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 10 11:45:04.027935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64575460-ae8e-45ef-8614-4c94307c8f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:04.028595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575460-ae8e-45ef-8614-4c94307c8f68 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:04.028879 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575460-ae8e-45ef-8614-4c94307c8f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:04.029155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575460-ae8e-45ef-8614-4c94307c8f68 None admin] PATH_INFO: `/v3/users/10d738bcb5484a248894a50e95518057` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:04.030537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64575460-ae8e-45ef-8614-4c94307c8f68 None admin] RBAC: Authorizing `identity:get_user(user_id=10d738bcb5484a248894a50e95518057)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:04.032029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64575460-ae8e-45ef-8614-4c94307c8f68 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:04.033610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 19/39] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:03 2022] GET /identity/v3/users/10d738bcb5484a248894a50e95518057 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:04.039262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1c18b-2cff-4d0a-bbe6-b92a270a09c3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:04.071867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ceef7-7060-4151-8e48-12365b208a4d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:04.072154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ceef7-7060-4151-8e48-12365b208a4d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:04.072444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ceef7-7060-4151-8e48-12365b208a4d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:04.073773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 20/40] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:04.078621 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d836f-4811-4af9-a536-36cc458a098e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:04.079011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d836f-4811-4af9-a536-36cc458a098e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:04.079295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d836f-4811-4af9-a536-36cc458a098e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:04.099482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-790d836f-4811-4af9-a536-36cc458a098e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:04.101362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-790d836f-4811-4af9-a536-36cc458a098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:04.142298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1c18b-2cff-4d0a-bbe6-b92a270a09c3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:04.149986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:04.174952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 21/41] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:04.179318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fe3d0-1a0a-4fc4-b9e0-39d746f96a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:04.180013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fe3d0-1a0a-4fc4-b9e0-39d746f96a00 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:04.180295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fe3d0-1a0a-4fc4-b9e0-39d746f96a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:04.180565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fe3d0-1a0a-4fc4-b9e0-39d746f96a00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:04.182264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fe3d0-1a0a-4fc4-b9e0-39d746f96a00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:04.183872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fe3d0-1a0a-4fc4-b9e0-39d746f96a00 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:04.189944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 21/42] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:04.193846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d836f-4811-4af9-a536-36cc458a098e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:04.203525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:04.288894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fe3d0-1a0a-4fc4-b9e0-39d746f96a00 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:04.292576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d836f-4811-4af9-a536-36cc458a098e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:04.326405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362fc52-88b5-4ca9-bb8f-c95611857f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:04.327095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362fc52-88b5-4ca9-bb8f-c95611857f8c None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:04.327382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362fc52-88b5-4ca9-bb8f-c95611857f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:04.327656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362fc52-88b5-4ca9-bb8f-c95611857f8c None admin] PATH_INFO: `/v3/domains/default/users/10d738bcb5484a248894a50e95518057/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:04.329600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362fc52-88b5-4ca9-bb8f-c95611857f8c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=10d738bcb5484a248894a50e95518057, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:04.331058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362fc52-88b5-4ca9-bb8f-c95611857f8c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:04.350412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 22/43] 10.222.0.52 () {58 vars in 1361 bytes} [Mon Oct 10 11:45:04 2022] PUT /identity/v3/domains/default/users/10d738bcb5484a248894a50e95518057/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:45:04.400583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fe3d0-1a0a-4fc4-b9e0-39d746f96a00 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:04.443504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be7e38-7eca-43ec-b55c-0dd7f22e67a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:04.444162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be7e38-7eca-43ec-b55c-0dd7f22e67a1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:04.444444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be7e38-7eca-43ec-b55c-0dd7f22e67a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:04.444720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be7e38-7eca-43ec-b55c-0dd7f22e67a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:04.445909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be7e38-7eca-43ec-b55c-0dd7f22e67a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:04.448031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be7e38-7eca-43ec-b55c-0dd7f22e67a1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:04.449914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 22/44] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:04.455396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362fc52-88b5-4ca9-bb8f-c95611857f8c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:04.465344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be7e38-7eca-43ec-b55c-0dd7f22e67a1 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:04.513669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:04.541280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:04.579752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362fc52-88b5-4ca9-bb8f-c95611857f8c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:04.582115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be7e38-7eca-43ec-b55c-0dd7f22e67a1 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:04.615450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb29f08-659e-4098-b722-ea5dc7a68c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:04.616067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb29f08-659e-4098-b722-ea5dc7a68c8a None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:04.616353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb29f08-659e-4098-b722-ea5dc7a68c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:04.616629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb29f08-659e-4098-b722-ea5dc7a68c8a None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:04.617295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb29f08-659e-4098-b722-ea5dc7a68c8a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:04.618828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051247b-4ee3-48f0-aaf3-48b98a86689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:04.619314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb29f08-659e-4098-b722-ea5dc7a68c8a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:04.619498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051247b-4ee3-48f0-aaf3-48b98a86689d None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:04.619754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051247b-4ee3-48f0-aaf3-48b98a86689d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:04.620039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051247b-4ee3-48f0-aaf3-48b98a86689d None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:04.620723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051247b-4ee3-48f0-aaf3-48b98a86689d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:04.622101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051247b-4ee3-48f0-aaf3-48b98a86689d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:04.636905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 23/45] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Oct 10 11:45:04 2022] POST /identity/v3/roles => generated 223 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:45:04.651728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:45:04 2022] POST /identity/v3/projects => generated 312 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:06.499465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0d35-a5c7-498e-a271-076ca811f5c6 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:06.499963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0d35-a5c7-498e-a271-076ca811f5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:06.500070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0d35-a5c7-498e-a271-076ca811f5c6 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:06.501356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 24/47] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:06.506604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6464e-b7db-4666-ad38-a2ca69b5ab45 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:06.507019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6464e-b7db-4666-ad38-a2ca69b5ab45 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:06.507173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6464e-b7db-4666-ad38-a2ca69b5ab45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:06.525310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5db6464e-b7db-4666-ad38-a2ca69b5ab45 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:06.526483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5db6464e-b7db-4666-ad38-a2ca69b5ab45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:06.590885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 24/48] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:06.822613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28fb0-c897-4f3b-9623-ec3a9c51ce52 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:06.823042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28fb0-c897-4f3b-9623-ec3a9c51ce52 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:06.823320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28fb0-c897-4f3b-9623-ec3a9c51ce52 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:06.824659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 25/49] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:06.830361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90f52-bea8-4576-924e-b0e582dcc8bf None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:06.830651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90f52-bea8-4576-924e-b0e582dcc8bf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:06.830943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90f52-bea8-4576-924e-b0e582dcc8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:06.849504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bcd90f52-bea8-4576-924e-b0e582dcc8bf None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:06.850680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bcd90f52-bea8-4576-924e-b0e582dcc8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:06.867813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f28fb0-c897-4f3b-9623-ec3a9c51ce52 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:06.897597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:06.920243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 25/50] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:06.969665 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f28fb0-c897-4f3b-9623-ec3a9c51ce52 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:06.971302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:07.005247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1af0c-a911-49a1-a906-d14815c5360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:07.005868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1af0c-a911-49a1-a906-d14815c5360e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:07.006148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1af0c-a911-49a1-a906-d14815c5360e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:07.006425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1af0c-a911-49a1-a906-d14815c5360e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:07.016710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1af0c-a911-49a1-a906-d14815c5360e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-47d1af0c-a911-49a1-a906-d14815c5360e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:07.021469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:07.022416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:07.023385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 26/51] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:07.030568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd90f52-bea8-4576-924e-b0e582dcc8bf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:07.128419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd90f52-bea8-4576-924e-b0e582dcc8bf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:07.189639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1af0c-a911-49a1-a906-d14815c5360e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:07.266225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:07.266827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:07.267134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:07.267404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:07.268065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:07.268362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:07.269707 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:07.278109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 26/52] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:07.283826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:07.291181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1af0c-a911-49a1-a906-d14815c5360e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:07.324001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39d421-0813-42f0-a522-d8bfc38b58f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:07.324626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39d421-0813-42f0-a522-d8bfc38b58f1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:07.324903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39d421-0813-42f0-a522-d8bfc38b58f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:07.325173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39d421-0813-42f0-a522-d8bfc38b58f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:07.325807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:07.326271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39d421-0813-42f0-a522-d8bfc38b58f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:07.328197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39d421-0813-42f0-a522-d8bfc38b58f1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:07.329992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:07.335447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39d421-0813-42f0-a522-d8bfc38b58f1 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:07.365507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:07.382432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d645-25a9-4e52-805b-8c7e4267eca5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:07.421294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca497f-dd53-4430-b3fe-25fc4cbad262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:07.421948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca497f-dd53-4430-b3fe-25fc4cbad262 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:07.422241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca497f-dd53-4430-b3fe-25fc4cbad262 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:07.422520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca497f-dd53-4430-b3fe-25fc4cbad262 None admin] PATH_INFO: `/v3/users/10d738bcb5484a248894a50e95518057` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:07.423713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca497f-dd53-4430-b3fe-25fc4cbad262 None admin] RBAC: Authorizing `identity:get_user(user_id=10d738bcb5484a248894a50e95518057)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:07.425273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca497f-dd53-4430-b3fe-25fc4cbad262 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:07.426871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 27/54] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:07 2022] GET /identity/v3/users/10d738bcb5484a248894a50e95518057 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:07.432799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca497f-dd53-4430-b3fe-25fc4cbad262 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:07.441812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39d421-0813-42f0-a522-d8bfc38b58f1 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:07.474619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849a6ba-51c5-4385-974b-6dcbb092960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:07.475266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a6ba-51c5-4385-974b-6dcbb092960c None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:07.475554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a6ba-51c5-4385-974b-6dcbb092960c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:07.475835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a6ba-51c5-4385-974b-6dcbb092960c None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:07.476495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849a6ba-51c5-4385-974b-6dcbb092960c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:07.477969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849a6ba-51c5-4385-974b-6dcbb092960c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:07.505790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 28/55] 10.222.0.52 () {60 vars in 1140 bytes} [Mon Oct 10 11:45:07 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:07.532784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca497f-dd53-4430-b3fe-25fc4cbad262 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:07.544342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:07.570273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1a21c-44e7-4ae1-83bd-907a16fcf995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:07.570949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1a21c-44e7-4ae1-83bd-907a16fcf995 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:07.571366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1a21c-44e7-4ae1-83bd-907a16fcf995 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:07.572127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1a21c-44e7-4ae1-83bd-907a16fcf995 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:07.573298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1a21c-44e7-4ae1-83bd-907a16fcf995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:07.574744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1a21c-44e7-4ae1-83bd-907a16fcf995 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:07.576388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 28/56] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:07.582438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849a6ba-51c5-4385-974b-6dcbb092960c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:07.666645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:07.690460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849a6ba-51c5-4385-974b-6dcbb092960c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:07.724367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ae7f-d2d9-4f4b-b890-5f9a09a05248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:07.724984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ae7f-d2d9-4f4b-b890-5f9a09a05248 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:07.725268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ae7f-d2d9-4f4b-b890-5f9a09a05248 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:07.725547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ae7f-d2d9-4f4b-b890-5f9a09a05248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:07.726277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3ae7f-d2d9-4f4b-b890-5f9a09a05248 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.domain.id=default, user.id=10d738bcb5484a248894a50e95518057) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:07.726587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3ae7f-d2d9-4f4b-b890-5f9a09a05248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:07.728616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3ae7f-d2d9-4f4b-b890-5f9a09a05248 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:07.734468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 29/57] 10.222.0.52 () {56 vars in 1322 bytes} [Mon Oct 10 11:45:07 2022] GET /identity/v3/role_assignments?user.id=10d738bcb5484a248894a50e95518057&scope.domain.id=default&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:09.660298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b409b-737c-4a09-a3f9-640c56616c51 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:09.660779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b409b-737c-4a09-a3f9-640c56616c51 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:09.660779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b409b-737c-4a09-a3f9-640c56616c51 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:09.662093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 29/58] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:09.667791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e170f-7d54-4365-bd8b-47fd33e992ee None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:09.668186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e170f-7d54-4365-bd8b-47fd33e992ee None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:09.668356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e170f-7d54-4365-bd8b-47fd33e992ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:09.686772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-784e170f-7d54-4365-bd8b-47fd33e992ee None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:09.688022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-784e170f-7d54-4365-bd8b-47fd33e992ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:09.752931 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 30/59] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:10.019301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b409b-737c-4a09-a3f9-640c56616c51 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:10.118438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b409b-737c-4a09-a3f9-640c56616c51 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:10.151162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711c7bb-a862-431e-ade6-775fa631bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:10.151769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711c7bb-a862-431e-ade6-775fa631bd6b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:10.152047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711c7bb-a862-431e-ade6-775fa631bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:10.152315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711c7bb-a862-431e-ade6-775fa631bd6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:10.153406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711c7bb-a862-431e-ade6-775fa631bd6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:10.154825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711c7bb-a862-431e-ade6-775fa631bd6b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:10.156366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 30/60] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:10.161702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e170f-7d54-4365-bd8b-47fd33e992ee None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:10.234427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:10.257348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e170f-7d54-4365-bd8b-47fd33e992ee None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:10.289377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eea33-2d2b-4466-a9a5-8c0d94805fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:10.289928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eea33-2d2b-4466-a9a5-8c0d94805fa3 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:10.290216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eea33-2d2b-4466-a9a5-8c0d94805fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:10.290486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eea33-2d2b-4466-a9a5-8c0d94805fa3 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:10.291172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110eea33-2d2b-4466-a9a5-8c0d94805fa3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:10.292603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110eea33-2d2b-4466-a9a5-8c0d94805fa3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:10.318760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 31/61] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:10 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:13.635215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ed7e-1e63-4a98-9151-91ebfe633ed8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:13.636127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ed7e-1e63-4a98-9151-91ebfe633ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:13.636580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ed7e-1e63-4a98-9151-91ebfe633ed8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:13.638430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 31/62] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:13.643431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b82a7-6743-4c41-8a5f-8615761383fc None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:13.644219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b82a7-6743-4c41-8a5f-8615761383fc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:13.644673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b82a7-6743-4c41-8a5f-8615761383fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:13.672249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-4b7b82a7-6743-4c41-8a5f-8615761383fc None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:13.673789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4b7b82a7-6743-4c41-8a5f-8615761383fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:13.763892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 32/63] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:13.827956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0fb9-4aa8-4c1b-ad98-5ce56d29601b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:13.828573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0fb9-4aa8-4c1b-ad98-5ce56d29601b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:13.829075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0fb9-4aa8-4c1b-ad98-5ce56d29601b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:13.830822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 32/64] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:13.836508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162fcc-d835-4b70-ad31-b45dcb2e8cf2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:13.837429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162fcc-d835-4b70-ad31-b45dcb2e8cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:13.837714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162fcc-d835-4b70-ad31-b45dcb2e8cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:13.859076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f5162fcc-d835-4b70-ad31-b45dcb2e8cf2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:13.860332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f5162fcc-d835-4b70-ad31-b45dcb2e8cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:13.944011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:13.961531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 33/65] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:13.979322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78cb33-fb28-438e-9740-03e23355d4f4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:13.979969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78cb33-fb28-438e-9740-03e23355d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:13.980447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78cb33-fb28-438e-9740-03e23355d4f4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:13.982040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 33/66] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:13.992786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ae8d-1a65-4252-8430-057ec6dda4a9 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:13.993662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ae8d-1a65-4252-8430-057ec6dda4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:13.993946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ae8d-1a65-4252-8430-057ec6dda4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.017217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c654ae8d-1a65-4252-8430-057ec6dda4a9 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:14.018418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c654ae8d-1a65-4252-8430-057ec6dda4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:14.093550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88519f7-c0b9-4977-9edc-a41c83bb8e4d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.094187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88519f7-c0b9-4977-9edc-a41c83bb8e4d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.094664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88519f7-c0b9-4977-9edc-a41c83bb8e4d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.096245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 34/67] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:14.101491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d8d39-5837-4033-97a6-9c6cd0d89483 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.102032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d8d39-5837-4033-97a6-9c6cd0d89483 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.102538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d8d39-5837-4033-97a6-9c6cd0d89483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.111491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 34/68] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:14.136438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bf5d8d39-5837-4033-97a6-9c6cd0d89483 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:14.137710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bf5d8d39-5837-4033-97a6-9c6cd0d89483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:14.165629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ae8d-1a65-4252-8430-057ec6dda4a9 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.212972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:14.254995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 35/69] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:14.256523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3e50-5aea-4cca-a2ec-5905c0792f5b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.256957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3e50-5aea-4cca-a2ec-5905c0792f5b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.257376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3e50-5aea-4cca-a2ec-5905c0792f5b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.263058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 36/70] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:14.264769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a5bcd-7ef1-4a61-8029-4d4f48fb11d2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.265204 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a5bcd-7ef1-4a61-8029-4d4f48fb11d2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.265625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a5bcd-7ef1-4a61-8029-4d4f48fb11d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.278519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ae8d-1a65-4252-8430-057ec6dda4a9 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.311141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:14.311887 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-486a5bcd-7ef1-4a61-8029-4d4f48fb11d2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:14.313265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-486a5bcd-7ef1-4a61-8029-4d4f48fb11d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:14.316972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709708c0-ae95-4fab-9f3b-a89c6f22d97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.317776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709708c0-ae95-4fab-9f3b-a89c6f22d97c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.318265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709708c0-ae95-4fab-9f3b-a89c6f22d97c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.318673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709708c0-ae95-4fab-9f3b-a89c6f22d97c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.332034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709708c0-ae95-4fab-9f3b-a89c6f22d97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-709708c0-ae95-4fab-9f3b-a89c6f22d97c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:14.337937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:14.339008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:14.339953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:14.342152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 35/71] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:14.348186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709708c0-ae95-4fab-9f3b-a89c6f22d97c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.411278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 37/72] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:14.415992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a5bcd-7ef1-4a61-8029-4d4f48fb11d2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.485675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709708c0-ae95-4fab-9f3b-a89c6f22d97c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.525083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a5bcd-7ef1-4a61-8029-4d4f48fb11d2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.532476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac7a5-6def-43ac-8665-2a716daff5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.533564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:14.534504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac7a5-6def-43ac-8665-2a716daff5fa None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.534948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac7a5-6def-43ac-8665-2a716daff5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.536658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac7a5-6def-43ac-8665-2a716daff5fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.549560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeac7a5-6def-43ac-8665-2a716daff5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-cbeac7a5-6def-43ac-8665-2a716daff5fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:14.556772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:14.557772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:14.558729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:14.561388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 36/73] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:14.562705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac7a5-6def-43ac-8665-2a716daff5fa None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.566347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.567116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.567633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.568101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.568955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:14.569447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:14.571617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:14.580438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 38/74] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:14.585998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.609464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:14.665854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac7a5-6def-43ac-8665-2a716daff5fa None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.700092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0e6c9-11c0-4367-9053-830b8a4ce7af None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.702112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109ee5f-eba4-46e1-a716-b137997d503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.702979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109ee5f-eba4-46e1-a716-b137997d503c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.703456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109ee5f-eba4-46e1-a716-b137997d503c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.703927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109ee5f-eba4-46e1-a716-b137997d503c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.714789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109ee5f-eba4-46e1-a716-b137997d503c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-8109ee5f-eba4-46e1-a716-b137997d503c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:14.719676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:14.720503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 37/75] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:14.721601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109ee5f-eba4-46e1-a716-b137997d503c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.736149 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba2069-0ca4-487d-8764-cd4930c746f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.736719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba2069-0ca4-487d-8764-cd4930c746f1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.737004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba2069-0ca4-487d-8764-cd4930c746f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.737273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba2069-0ca4-487d-8764-cd4930c746f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.738399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba2069-0ca4-487d-8764-cd4930c746f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:14.739852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba2069-0ca4-487d-8764-cd4930c746f1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:14.741407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 39/76] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:14.741407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba2069-0ca4-487d-8764-cd4930c746f1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.777955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:14.819927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109ee5f-eba4-46e1-a716-b137997d503c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.822283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:14.847715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba2069-0ca4-487d-8764-cd4930c746f1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.857105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.857718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.858003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.858279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.858980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:14.859289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:14.860689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:14.869160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 38/77] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:14.869941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.888530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f8ce-4d4b-40c5-a1b1-e927eb43caa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.889198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f8ce-4d4b-40c5-a1b1-e927eb43caa9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.889477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f8ce-4d4b-40c5-a1b1-e927eb43caa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.889745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f8ce-4d4b-40c5-a1b1-e927eb43caa9 None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.904220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3f8ce-4d4b-40c5-a1b1-e927eb43caa9 None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:14.905644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3f8ce-4d4b-40c5-a1b1-e927eb43caa9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:14.907256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 40/78] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:14.908010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f8ce-4d4b-40c5-a1b1-e927eb43caa9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.970959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef247d45-341b-42af-8f59-7a34c7753205 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.998744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:15.004593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43847ca8-de14-4012-baa5-d4e8b5e94ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.005223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43847ca8-de14-4012-baa5-d4e8b5e94ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.005440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f8ce-4d4b-40c5-a1b1-e927eb43caa9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.005563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43847ca8-de14-4012-baa5-d4e8b5e94ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.005760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43847ca8-de14-4012-baa5-d4e8b5e94ab2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.016273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43847ca8-de14-4012-baa5-d4e8b5e94ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-43847ca8-de14-4012-baa5-d4e8b5e94ab2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:15.022114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:15.022992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:15.023757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 39/79] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:15.024988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43847ca8-de14-4012-baa5-d4e8b5e94ab2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.040463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.041073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.041359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.041636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.042298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:15.042600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.044020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.051462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:15.057230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 41/80] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:15.057230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.127483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43847ca8-de14-4012-baa5-d4e8b5e94ab2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.151620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fd305-4370-4e74-81b7-b7710c9876f8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.160306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f913-3e97-47f9-a852-289c53891487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.160912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f913-3e97-47f9-a852-289c53891487 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.161193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f913-3e97-47f9-a852-289c53891487 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.161464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f913-3e97-47f9-a852-289c53891487 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.162106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131f913-3e97-47f9-a852-289c53891487 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.163598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131f913-3e97-47f9-a852-289c53891487 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.184386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 40/81] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:15 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:15.185183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f913-3e97-47f9-a852-289c53891487 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.188517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de009499-78c9-4b2b-92e1-ac2c3821ce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.189249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009499-78c9-4b2b-92e1-ac2c3821ce7f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.189619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009499-78c9-4b2b-92e1-ac2c3821ce7f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.190166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009499-78c9-4b2b-92e1-ac2c3821ce7f None admin] PATH_INFO: `/v3/users/10d738bcb5484a248894a50e95518057` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.191405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de009499-78c9-4b2b-92e1-ac2c3821ce7f None admin] RBAC: Authorizing `identity:get_user(user_id=10d738bcb5484a248894a50e95518057)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.192949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de009499-78c9-4b2b-92e1-ac2c3821ce7f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.194602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 42/82] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/users/10d738bcb5484a248894a50e95518057 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:15.195434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de009499-78c9-4b2b-92e1-ac2c3821ce7f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.228129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:15.286537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:15.310315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f913-3e97-47f9-a852-289c53891487 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.318433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de009499-78c9-4b2b-92e1-ac2c3821ce7f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.349097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f6410-5100-4a8a-981e-90ca13091014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.349763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f6410-5100-4a8a-981e-90ca13091014 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.350048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f6410-5100-4a8a-981e-90ca13091014 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.350325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f6410-5100-4a8a-981e-90ca13091014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.361558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.362174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.362453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.362728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.363715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:15.363822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.365208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.377622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 43/83] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:15.378360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.434192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f6410-5100-4a8a-981e-90ca13091014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.435695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f6410-5100-4a8a-981e-90ca13091014 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.501274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:15.519344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af0f3d-814e-4f01-9afc-b19ce4dc1abf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.538797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:15.563862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 41/84] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:15.566209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f6410-5100-4a8a-981e-90ca13091014 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.572259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928a288-82ce-4c1d-abed-9dc8d6f33da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.572538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928a288-82ce-4c1d-abed-9dc8d6f33da3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.572763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928a288-82ce-4c1d-abed-9dc8d6f33da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.573038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928a288-82ce-4c1d-abed-9dc8d6f33da3 None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.574203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928a288-82ce-4c1d-abed-9dc8d6f33da3 None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.575670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928a288-82ce-4c1d-abed-9dc8d6f33da3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.577400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 44/85] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:15.578010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928a288-82ce-4c1d-abed-9dc8d6f33da3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.678092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f6410-5100-4a8a-981e-90ca13091014 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.693543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928a288-82ce-4c1d-abed-9dc8d6f33da3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.714630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48389ad-8991-4579-bc85-f18fdd37fa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.715376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48389ad-8991-4579-bc85-f18fdd37fa2c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.715658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48389ad-8991-4579-bc85-f18fdd37fa2c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.715932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48389ad-8991-4579-bc85-f18fdd37fa2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.730319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8386b53-aeeb-4851-9df7-732aa17077ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.730979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8386b53-aeeb-4851-9df7-732aa17077ea None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.731338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8386b53-aeeb-4851-9df7-732aa17077ea None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.731745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8386b53-aeeb-4851-9df7-732aa17077ea None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.732922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8386b53-aeeb-4851-9df7-732aa17077ea None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.734409 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8386b53-aeeb-4851-9df7-732aa17077ea None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.736618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 45/86] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:15.737399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8386b53-aeeb-4851-9df7-732aa17077ea None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.763770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:15.772375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:15.795546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48389ad-8991-4579-bc85-f18fdd37fa2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.797100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48389ad-8991-4579-bc85-f18fdd37fa2c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.851678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8386b53-aeeb-4851-9df7-732aa17077ea None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:15.889567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7cceb-cca6-4851-838c-3cd6e117acc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:15.890206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7cceb-cca6-4851-838c-3cd6e117acc9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:15.890486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7cceb-cca6-4851-838c-3cd6e117acc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:15.890759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7cceb-cca6-4851-838c-3cd6e117acc9 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:15.891910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7cceb-cca6-4851-838c-3cd6e117acc9 None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:15.893787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7cceb-cca6-4851-838c-3cd6e117acc9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:15.895984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 46/87] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:15.901451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7cceb-cca6-4851-838c-3cd6e117acc9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.906131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 42/88] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:15.907012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48389ad-8991-4579-bc85-f18fdd37fa2c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:15.994193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:16.002921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:16.015397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7cceb-cca6-4851-838c-3cd6e117acc9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:16.016769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48389ad-8991-4579-bc85-f18fdd37fa2c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:16.052386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38fc6c-b5c2-4dd6-8646-16a09c0f53c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:16.052917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38fc6c-b5c2-4dd6-8646-16a09c0f53c6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.053246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38fc6c-b5c2-4dd6-8646-16a09c0f53c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.053520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38fc6c-b5c2-4dd6-8646-16a09c0f53c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.053668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df740f7-9b4e-4913-926b-e16a2e03700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:16.054414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df740f7-9b4e-4913-926b-e16a2e03700c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.054843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df740f7-9b4e-4913-926b-e16a2e03700c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.055315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df740f7-9b4e-4913-926b-e16a2e03700c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.129930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df740f7-9b4e-4913-926b-e16a2e03700c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:16.130920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38fc6c-b5c2-4dd6-8646-16a09c0f53c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:16.131812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df740f7-9b4e-4913-926b-e16a2e03700c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:16.132459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38fc6c-b5c2-4dd6-8646-16a09c0f53c6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:16.217123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:16.231579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:16.240756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 43/89] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:16.241855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df740f7-9b4e-4913-926b-e16a2e03700c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:16.243785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 47/90] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:16.248289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38fc6c-b5c2-4dd6-8646-16a09c0f53c6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:16.350210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df740f7-9b4e-4913-926b-e16a2e03700c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:16.352323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38fc6c-b5c2-4dd6-8646-16a09c0f53c6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:16.385035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:16.385831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.386276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.386686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.387576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] RBAC: Adding query filter params (role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db, user.id=4dbfa2988ba9444197b16560d6e65096) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:16.388027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461da8d5-9b46-4ef5-b9e7-66b0d724a17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:16.388147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:16.388832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461da8d5-9b46-4ef5-b9e7-66b0d724a17e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.389210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461da8d5-9b46-4ef5-b9e7-66b0d724a17e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.389580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461da8d5-9b46-4ef5-b9e7-66b0d724a17e None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.390176 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:16.391039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461da8d5-9b46-4ef5-b9e7-66b0d724a17e None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:16.392874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461da8d5-9b46-4ef5-b9e7-66b0d724a17e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:16.394685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 48/91] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:16 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:16.395668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461da8d5-9b46-4ef5-b9e7-66b0d724a17e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:16.402922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 44/92] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:16 2022] GET /identity/v3/role_assignments?user.id=4dbfa2988ba9444197b16560d6e65096&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:16.402922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:16.456334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:16.515351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:16.539029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461da8d5-9b46-4ef5-b9e7-66b0d724a17e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:16.559389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71448ad7-ee9c-41ee-a23a-b44fabc82c8a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:16.577632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd866e14-d586-4d0c-aee2-c4463caddf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:16.578670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd866e14-d586-4d0c-aee2-c4463caddf9b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.579390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd866e14-d586-4d0c-aee2-c4463caddf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.579976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd866e14-d586-4d0c-aee2-c4463caddf9b None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.581442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd866e14-d586-4d0c-aee2-c4463caddf9b None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:16.583228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd866e14-d586-4d0c-aee2-c4463caddf9b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:16.585151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 49/93] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:16 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:16.586122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd866e14-d586-4d0c-aee2-c4463caddf9b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:16.607180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b8a53-2b4f-4b02-b48c-b550449542ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:16.607747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b8a53-2b4f-4b02-b48c-b550449542ae None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.608030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b8a53-2b4f-4b02-b48c-b550449542ae None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.608311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b8a53-2b4f-4b02-b48c-b550449542ae None admin] PATH_INFO: `/v3/projects/5849148002994ca99651e7026aa6c431` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.609492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b8a53-2b4f-4b02-b48c-b550449542ae None admin] RBAC: Authorizing `identity:get_project(project_id=5849148002994ca99651e7026aa6c431)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:16.611016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b8a53-2b4f-4b02-b48c-b550449542ae None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:16.619362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 45/94] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:16 2022] GET /identity/v3/projects/5849148002994ca99651e7026aa6c431 => generated 312 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:16.619362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b8a53-2b4f-4b02-b48c-b550449542ae None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:16.724482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b8a53-2b4f-4b02-b48c-b550449542ae None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:16.737543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:16.764817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:16.765428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.765706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.765974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.766697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] RBAC: Adding query filter params (role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db, user.id=4dbfa2988ba9444197b16560d6e65096) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:16.767030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:16.768470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:16.783962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 46/95] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:16 2022] GET /identity/v3/role_assignments?user.id=4dbfa2988ba9444197b16560d6e65096&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=2e40648646094078acac22f8eb94dd98 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:16.784786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:16.809151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd866e14-d586-4d0c-aee2-c4463caddf9b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:16.868631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:16.895207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ef290-72f7-48cd-b4a7-16d4cbf7ed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:16.895831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef290-72f7-48cd-b4a7-16d4cbf7ed58 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.896114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef290-72f7-48cd-b4a7-16d4cbf7ed58 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.896389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef290-72f7-48cd-b4a7-16d4cbf7ed58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.897294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ef290-72f7-48cd-b4a7-16d4cbf7ed58 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db, user.id=10d738bcb5484a248894a50e95518057) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:16.897610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ef290-72f7-48cd-b4a7-16d4cbf7ed58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:16.900101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ef290-72f7-48cd-b4a7-16d4cbf7ed58 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:16.919243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 50/96] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:16 2022] GET /identity/v3/role_assignments?user.id=10d738bcb5484a248894a50e95518057&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:17.001868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9a80d-90b4-4b3d-80cf-3bc1475207d7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:17.053017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52423712-a1b3-4da8-9f91-7d1e173a3116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:17.053981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423712-a1b3-4da8-9f91-7d1e173a3116 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:17.054457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423712-a1b3-4da8-9f91-7d1e173a3116 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:17.054968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423712-a1b3-4da8-9f91-7d1e173a3116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:17.055963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52423712-a1b3-4da8-9f91-7d1e173a3116 None admin] RBAC: Adding query filter params (role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=5849148002994ca99651e7026aa6c431, user.id=4dbfa2988ba9444197b16560d6e65096) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:17.056678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52423712-a1b3-4da8-9f91-7d1e173a3116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:17.058374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52423712-a1b3-4da8-9f91-7d1e173a3116 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:17.066636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 47/97] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:16 2022] GET /identity/v3/role_assignments?user.id=4dbfa2988ba9444197b16560d6e65096&scope.project.id=5849148002994ca99651e7026aa6c431&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:17.728148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e3ce-c0a4-484f-b682-e8b23bd1b016 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:17.728359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e3ce-c0a4-484f-b682-e8b23bd1b016 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:17.728880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e3ce-c0a4-484f-b682-e8b23bd1b016 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:17.730410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 51/98] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:17.735844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88103944-1243-4fea-985d-1701a4572c38 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:17.736290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88103944-1243-4fea-985d-1701a4572c38 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:17.736462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88103944-1243-4fea-985d-1701a4572c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:17.764018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-88103944-1243-4fea-985d-1701a4572c38 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:17.765255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-88103944-1243-4fea-985d-1701a4572c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:17.810571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:17.859835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 48/99] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:18.082922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1e31-50ea-4780-b981-c5db87aaeacc None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:18.082922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1e31-50ea-4780-b981-c5db87aaeacc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:18.082922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1e31-50ea-4780-b981-c5db87aaeacc None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:18.084052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 52/100] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:18.089715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7d0d-5320-4995-bd13-05a3527e8bdd None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:18.090016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7d0d-5320-4995-bd13-05a3527e8bdd None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:18.090296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7d0d-5320-4995-bd13-05a3527e8bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:18.116084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-1b9d7d0d-5320-4995-bd13-05a3527e8bdd None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:18.117353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1b9d7d0d-5320-4995-bd13-05a3527e8bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:18.185321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 49/101] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:18.428665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba1e31-50ea-4780-b981-c5db87aaeacc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:18.566854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba1e31-50ea-4780-b981-c5db87aaeacc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:18.608983 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93f1a4-cf49-4725-9d3d-bc0ddecc7733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:18.609929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93f1a4-cf49-4725-9d3d-bc0ddecc7733 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:18.610407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93f1a4-cf49-4725-9d3d-bc0ddecc7733 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:18.610866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93f1a4-cf49-4725-9d3d-bc0ddecc7733 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:18.612176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93f1a4-cf49-4725-9d3d-bc0ddecc7733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:18.613822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93f1a4-cf49-4725-9d3d-bc0ddecc7733 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:18.615708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 53/102] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:18.621671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d7d0d-5320-4995-bd13-05a3527e8bdd None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:18.702276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93f1a4-cf49-4725-9d3d-bc0ddecc7733 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:18.770915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:18.777644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d7d0d-5320-4995-bd13-05a3527e8bdd None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:18.843259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:18.862553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faa615-e152-413a-bbc4-4646fe99e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:18.868836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faa615-e152-413a-bbc4-4646fe99e563 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:18.869344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faa615-e152-413a-bbc4-4646fe99e563 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:18.869833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faa615-e152-413a-bbc4-4646fe99e563 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:18.870673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faa615-e152-413a-bbc4-4646fe99e563 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:18.872343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faa615-e152-413a-bbc4-4646fe99e563 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:18.919374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 50/103] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:45:18 2022] POST /identity/v3/projects => generated 302 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:18.931551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93f1a4-cf49-4725-9d3d-bc0ddecc7733 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:18.997193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042737e5-10e7-48e0-87bc-65ab3d38d9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:18.998279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042737e5-10e7-48e0-87bc-65ab3d38d9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:18.998770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042737e5-10e7-48e0-87bc-65ab3d38d9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:18.999304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042737e5-10e7-48e0-87bc-65ab3d38d9e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.021489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042737e5-10e7-48e0-87bc-65ab3d38d9e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-042737e5-10e7-48e0-87bc-65ab3d38d9e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:19.028425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:19.029358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:19.030119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:19.032999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 54/104] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:19.038497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faa615-e152-413a-bbc4-4646fe99e563 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:19.167142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faa615-e152-413a-bbc4-4646fe99e563 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:19.202362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:19.203325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.203829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.204291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.205174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:19.205665 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:19.207262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:19.216041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 51/105] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:19.221934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042737e5-10e7-48e0-87bc-65ab3d38d9e4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:19.355314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042737e5-10e7-48e0-87bc-65ab3d38d9e4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:19.381514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:19.404665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc257b-d931-4954-9d49-29f7754f6087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:19.405319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc257b-d931-4954-9d49-29f7754f6087 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.405599 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc257b-d931-4954-9d49-29f7754f6087 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.405876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc257b-d931-4954-9d49-29f7754f6087 None admin] PATH_INFO: `/v3/users/a6c35083aa25492e984904fd3fb9f394` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.432019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc257b-d931-4954-9d49-29f7754f6087 None admin] RBAC: Authorizing `identity:get_user(user_id=a6c35083aa25492e984904fd3fb9f394)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:19.433499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc257b-d931-4954-9d49-29f7754f6087 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:19.435231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 55/106] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:19 2022] GET /identity/v3/users/a6c35083aa25492e984904fd3fb9f394 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:19.440993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:19.471520 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:19.578719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caa45d-d1df-4b55-98f1-03acb5729819 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:19.627555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb6791-2a62-446a-84f8-47de45ef2d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:19.628140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb6791-2a62-446a-84f8-47de45ef2d4f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.628431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb6791-2a62-446a-84f8-47de45ef2d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.628698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb6791-2a62-446a-84f8-47de45ef2d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.722004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb6791-2a62-446a-84f8-47de45ef2d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:19.723600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb6791-2a62-446a-84f8-47de45ef2d4f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:19.748024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:19.851072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c78fef-7b4f-49d8-b00f-3a673d8aa766 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.851388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c78fef-7b4f-49d8-b00f-3a673d8aa766 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.851637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c78fef-7b4f-49d8-b00f-3a673d8aa766 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.853023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 56/107] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:19.857211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 52/108] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:19.858517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e41bc6-e054-4513-9d2c-01a0520c3b89 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.858808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e41bc6-e054-4513-9d2c-01a0520c3b89 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.859125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e41bc6-e054-4513-9d2c-01a0520c3b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.862620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb6791-2a62-446a-84f8-47de45ef2d4f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:19.880224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-31e41bc6-e054-4513-9d2c-01a0520c3b89 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:19.881351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-31e41bc6-e054-4513-9d2c-01a0520c3b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:19.973166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 57/109] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:19.974432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234b844-42cd-409a-9bbc-d7e36f5f511b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.974721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234b844-42cd-409a-9bbc-d7e36f5f511b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.979342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234b844-42cd-409a-9bbc-d7e36f5f511b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.981373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 58/110] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:19.987403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8263b3f-8b3f-491e-982f-5e6dc524250e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.987688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8263b3f-8b3f-491e-982f-5e6dc524250e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.987962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8263b3f-8b3f-491e-982f-5e6dc524250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.030914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a8263b3f-8b3f-491e-982f-5e6dc524250e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:20.032829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a8263b3f-8b3f-491e-982f-5e6dc524250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:20.080005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb6791-2a62-446a-84f8-47de45ef2d4f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.120400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:20.193427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c0f1-c8e5-4454-95a9-72af0803648b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.194446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c0f1-c8e5-4454-95a9-72af0803648b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.194972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c0f1-c8e5-4454-95a9-72af0803648b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.195489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c0f1-c8e5-4454-95a9-72af0803648b None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.199598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0c0f1-c8e5-4454-95a9-72af0803648b None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.201282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0c0f1-c8e5-4454-95a9-72af0803648b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.203370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 53/111] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:19 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:20.209798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c0f1-c8e5-4454-95a9-72af0803648b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.214570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 59/112] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:20.287000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0b5b-8d54-433d-825c-b9992b373048 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.287608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0b5b-8d54-433d-825c-b9992b373048 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.287608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0b5b-8d54-433d-825c-b9992b373048 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.288884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 60/113] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:20.300130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f44f2-5b82-4e1e-bf4a-e0096e780cfe None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.300426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f44f2-5b82-4e1e-bf4a-e0096e780cfe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.300708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f44f2-5b82-4e1e-bf4a-e0096e780cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.338934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-af4f44f2-5b82-4e1e-bf4a-e0096e780cfe None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:20.340157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-af4f44f2-5b82-4e1e-bf4a-e0096e780cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:20.379154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c0f1-c8e5-4454-95a9-72af0803648b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.432664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:20.487034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.487782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.488171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.488552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.489392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] RBAC: Adding query filter params (role.id=8aad1777fec94e0b8c3dc239ae4864c6, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db, user.id=a6c35083aa25492e984904fd3fb9f394) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:20.489812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.494917 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.503131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 54/114] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/role_assignments?user.id=a6c35083aa25492e984904fd3fb9f394&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=8aad1777fec94e0b8c3dc239ae4864c6 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:20.525851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.530429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 61/115] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:20.538940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f5e0-ce3a-45ee-9b61-908ab566bd42 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.538940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f5e0-ce3a-45ee-9b61-908ab566bd42 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.538940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f5e0-ce3a-45ee-9b61-908ab566bd42 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.540977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 62/116] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:20.546470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad34e4-e7f2-4dca-a43a-354c4edab2af None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.546756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad34e4-e7f2-4dca-a43a-354c4edab2af None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.547062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad34e4-e7f2-4dca-a43a-354c4edab2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.592123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a1ad34e4-e7f2-4dca-a43a-354c4edab2af None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:20.593396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a1ad34e4-e7f2-4dca-a43a-354c4edab2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:20.620578 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:20.738228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4eb9e-dd29-463a-a4f8-ef36f292b8da None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.740521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 63/117] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:20.741598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad34e4-e7f2-4dca-a43a-354c4edab2af None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.783124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d655f-8f70-4106-b68c-ac3911ce4d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.783764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d655f-8f70-4106-b68c-ac3911ce4d57 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.784044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d655f-8f70-4106-b68c-ac3911ce4d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.784312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d655f-8f70-4106-b68c-ac3911ce4d57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.818324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d655f-8f70-4106-b68c-ac3911ce4d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-7f2d655f-8f70-4106-b68c-ac3911ce4d57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:20.831353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:20.832215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:20.832966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 55/118] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:20.841125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d655f-8f70-4106-b68c-ac3911ce4d57 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.933725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad34e4-e7f2-4dca-a43a-354c4edab2af None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.951107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:20.994175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a5192-3717-44a6-ba9e-0183350fd610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.994855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a5192-3717-44a6-ba9e-0183350fd610 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.995156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a5192-3717-44a6-ba9e-0183350fd610 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.995426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a5192-3717-44a6-ba9e-0183350fd610 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.019633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a5192-3717-44a6-ba9e-0183350fd610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5f2a5192-3717-44a6-ba9e-0183350fd610 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:21.026327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:21.027231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:21.027988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 64/119] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:21.028601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a5192-3717-44a6-ba9e-0183350fd610 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.072879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d655f-8f70-4106-b68c-ac3911ce4d57 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.083004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:21.115735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.118940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.119512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.120009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.121068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:21.121570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:21.123247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:21.132730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 56/120] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:21.139060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.155740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a5192-3717-44a6-ba9e-0183350fd610 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.201001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa727c16-1c7e-43cc-96cf-378e9434bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.201890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa727c16-1c7e-43cc-96cf-378e9434bbdf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.202173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa727c16-1c7e-43cc-96cf-378e9434bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.202444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa727c16-1c7e-43cc-96cf-378e9434bbdf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.215618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa727c16-1c7e-43cc-96cf-378e9434bbdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-aa727c16-1c7e-43cc-96cf-378e9434bbdf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:21.222991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:21.223856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:21.224591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 65/121] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:21.225210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa727c16-1c7e-43cc-96cf-378e9434bbdf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.248466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:21.261392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f53c-c3e8-4f7b-a9a6-c4f8f5336281 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.306411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.307079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.307356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.307629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.308296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:21.308601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:21.310652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:21.323024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 57/122] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:21.327130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.353123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa727c16-1c7e-43cc-96cf-378e9434bbdf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.369233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:21.394769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7954e-4f45-479c-b78d-e28fd5069987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.395900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7954e-4f45-479c-b78d-e28fd5069987 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.396373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7954e-4f45-479c-b78d-e28fd5069987 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.396833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7954e-4f45-479c-b78d-e28fd5069987 None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.398265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7954e-4f45-479c-b78d-e28fd5069987 None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:21.400186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7954e-4f45-479c-b78d-e28fd5069987 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:21.402133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 66/123] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:21.403146 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7954e-4f45-479c-b78d-e28fd5069987 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.468525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a1599-09c1-4b63-9335-fbbdad687b98 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.511684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:21.511845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11533c9d-2f56-4985-b836-c5f5cd12c108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.512695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11533c9d-2f56-4985-b836-c5f5cd12c108 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.512798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11533c9d-2f56-4985-b836-c5f5cd12c108 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.513036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11533c9d-2f56-4985-b836-c5f5cd12c108 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.525612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11533c9d-2f56-4985-b836-c5f5cd12c108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-11533c9d-2f56-4985-b836-c5f5cd12c108 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:21.533476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:21.534376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:21.535598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:21.537831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 58/124] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:21.538139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7954e-4f45-479c-b78d-e28fd5069987 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.539376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11533c9d-2f56-4985-b836-c5f5cd12c108 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.586675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.587697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.588170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.588660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.589606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:21.590099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:21.591808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:21.601203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 67/125] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:21.602459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.650541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:21.661659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11533c9d-2f56-4985-b836-c5f5cd12c108 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.704270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38f569-a5db-4da8-a6e4-768e09fee69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.705293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38f569-a5db-4da8-a6e4-768e09fee69a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.705580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38f569-a5db-4da8-a6e4-768e09fee69a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.705858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38f569-a5db-4da8-a6e4-768e09fee69a None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.707299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38f569-a5db-4da8-a6e4-768e09fee69a None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:21.709239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38f569-a5db-4da8-a6e4-768e09fee69a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:21.711160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 59/126] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:21.712125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38f569-a5db-4da8-a6e4-768e09fee69a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.719489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d921b-ac3e-48fe-9825-abb76b41920d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.762479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f6734-30ab-4412-a954-83a553d1f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.763092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f6734-30ab-4412-a954-83a553d1f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.763380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f6734-30ab-4412-a954-83a553d1f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.763660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f6734-30ab-4412-a954-83a553d1f9b2 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.765493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f6734-30ab-4412-a954-83a553d1f9b2 None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:21.767128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f6734-30ab-4412-a954-83a553d1f9b2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:21.769942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 68/127] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:21.771115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f6734-30ab-4412-a954-83a553d1f9b2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.793530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:21.835302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38f569-a5db-4da8-a6e4-768e09fee69a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.880865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.880865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.880865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.880865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.881317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:21.881317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:21.881317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:21.889004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 60/128] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:21.894568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:21.899226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f6734-30ab-4412-a954-83a553d1f9b2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:21.919030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:21.941174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d7e6e-e233-4cf0-ae16-23f8779baa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:21.941814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d7e6e-e233-4cf0-ae16-23f8779baa85 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:21.942100 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d7e6e-e233-4cf0-ae16-23f8779baa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:21.942378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d7e6e-e233-4cf0-ae16-23f8779baa85 None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:21.943536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d7e6e-e233-4cf0-ae16-23f8779baa85 None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:21.944985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d7e6e-e233-4cf0-ae16-23f8779baa85 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:21.951678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 69/129] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:21.951678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d7e6e-e233-4cf0-ae16-23f8779baa85 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:22.009953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4e8d3-eb05-4ee2-96f6-e8b30f9d6b67 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:22.035924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:22.050848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24bcf6-ffd5-479e-927c-eb0fb672c3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:22.051512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24bcf6-ffd5-479e-927c-eb0fb672c3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.051801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24bcf6-ffd5-479e-927c-eb0fb672c3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.052089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24bcf6-ffd5-479e-927c-eb0fb672c3b9 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.053203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24bcf6-ffd5-479e-927c-eb0fb672c3b9 None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:22.054689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24bcf6-ffd5-479e-927c-eb0fb672c3b9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:22.056112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d7e6e-e233-4cf0-ae16-23f8779baa85 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:22.057079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 61/130] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:21 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:22.059721 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24bcf6-ffd5-479e-927c-eb0fb672c3b9 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:22.091948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925125c5-c602-4fbc-a2ea-6e2124b42147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:22.092665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925125c5-c602-4fbc-a2ea-6e2124b42147 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.092957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925125c5-c602-4fbc-a2ea-6e2124b42147 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.093239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925125c5-c602-4fbc-a2ea-6e2124b42147 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/users/4dbfa2988ba9444197b16560d6e65096/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.095212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925125c5-c602-4fbc-a2ea-6e2124b42147 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bf2c16b2e8b435bbd4cf6885ae631db, user_id=4dbfa2988ba9444197b16560d6e65096, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:22.096700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925125c5-c602-4fbc-a2ea-6e2124b42147 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:22.110719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 70/131] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:21 2022] PUT /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/users/4dbfa2988ba9444197b16560d6e65096/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:22.112246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511df81-5060-4279-b89e-ca141284b519 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.112538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511df81-5060-4279-b89e-ca141284b519 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.112819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511df81-5060-4279-b89e-ca141284b519 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.114099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 71/132] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:22.114828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511df81-5060-4279-b89e-ca141284b519 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:22.169344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:22.188037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24bcf6-ffd5-479e-927c-eb0fb672c3b9 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:22.229552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3b4b9-b2a5-43e1-8b3c-3ef4a5f07804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:22.230675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3b4b9-b2a5-43e1-8b3c-3ef4a5f07804 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.231252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3b4b9-b2a5-43e1-8b3c-3ef4a5f07804 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.231775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3b4b9-b2a5-43e1-8b3c-3ef4a5f07804 None admin] PATH_INFO: `/v3/users/10d738bcb5484a248894a50e95518057` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.233234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3b4b9-b2a5-43e1-8b3c-3ef4a5f07804 None admin] RBAC: Authorizing `identity:get_user(user_id=10d738bcb5484a248894a50e95518057)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:22.235302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3b4b9-b2a5-43e1-8b3c-3ef4a5f07804 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:22.237213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 62/133] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:22 2022] GET /identity/v3/users/10d738bcb5484a248894a50e95518057 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:22.238415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3b4b9-b2a5-43e1-8b3c-3ef4a5f07804 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:22.259792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511df81-5060-4279-b89e-ca141284b519 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:22.310309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3b956-1dd5-46af-aeac-d84006ccd64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:22.310810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3b956-1dd5-46af-aeac-d84006ccd64f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.311127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3b956-1dd5-46af-aeac-d84006ccd64f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.311403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3b956-1dd5-46af-aeac-d84006ccd64f None admin] PATH_INFO: `/v3/projects/5849148002994ca99651e7026aa6c431` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.312644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3b956-1dd5-46af-aeac-d84006ccd64f None admin] RBAC: Authorizing `identity:get_project(project_id=5849148002994ca99651e7026aa6c431)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:22.314298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3b956-1dd5-46af-aeac-d84006ccd64f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:22.316806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 72/134] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:22 2022] GET /identity/v3/projects/5849148002994ca99651e7026aa6c431 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:22.318316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a8215-732c-4514-a5f4-587e8a91f7c5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.318617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a8215-732c-4514-a5f4-587e8a91f7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.318989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a8215-732c-4514-a5f4-587e8a91f7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.338459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c47a8215-732c-4514-a5f4-587e8a91f7c5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:22.339740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c47a8215-732c-4514-a5f4-587e8a91f7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:22.343820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:22.371285 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3b4b9-b2a5-43e1-8b3c-3ef4a5f07804 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:22.410897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa236ef8-cd5f-449b-8133-4704c793495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:22.411967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa236ef8-cd5f-449b-8133-4704c793495e None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.412454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa236ef8-cd5f-449b-8133-4704c793495e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.412935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa236ef8-cd5f-449b-8133-4704c793495e None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/users/4dbfa2988ba9444197b16560d6e65096/roles/2e40648646094078acac22f8eb94dd98` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.416585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa236ef8-cd5f-449b-8133-4704c793495e None admin] RBAC: Authorizing `identity:create_grant(project_id=3bf2c16b2e8b435bbd4cf6885ae631db, user_id=4dbfa2988ba9444197b16560d6e65096, role_id=2e40648646094078acac22f8eb94dd98)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:22.419191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa236ef8-cd5f-449b-8133-4704c793495e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:22.430005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 73/135] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:22.431579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a8215-732c-4514-a5f4-587e8a91f7c5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:22.439143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 63/136] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:22 2022] PUT /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/users/4dbfa2988ba9444197b16560d6e65096/roles/2e40648646094078acac22f8eb94dd98 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:22.440542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa236ef8-cd5f-449b-8133-4704c793495e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:22.463667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:22.567060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a8215-732c-4514-a5f4-587e8a91f7c5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:22.574383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa236ef8-cd5f-449b-8133-4704c793495e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:22.606024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80684bb9-378d-417c-9c9f-d8682fddff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:22.607046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80684bb9-378d-417c-9c9f-d8682fddff91 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.607546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80684bb9-378d-417c-9c9f-d8682fddff91 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.608208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80684bb9-378d-417c-9c9f-d8682fddff91 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.609545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80684bb9-378d-417c-9c9f-d8682fddff91 None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:22.611202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80684bb9-378d-417c-9c9f-d8682fddff91 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:22.613377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 74/137] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:22 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:22.620256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80684bb9-378d-417c-9c9f-d8682fddff91 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:22.631454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf06354-9bc0-4be2-ac77-8864362ca8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:22.632162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf06354-9bc0-4be2-ac77-8864362ca8f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.632451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf06354-9bc0-4be2-ac77-8864362ca8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.632728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf06354-9bc0-4be2-ac77-8864362ca8f0 None admin] PATH_INFO: `/v3/projects/5849148002994ca99651e7026aa6c431/users/4dbfa2988ba9444197b16560d6e65096/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.634966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:22.638327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf06354-9bc0-4be2-ac77-8864362ca8f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5849148002994ca99651e7026aa6c431, user_id=4dbfa2988ba9444197b16560d6e65096, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:22.640046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf06354-9bc0-4be2-ac77-8864362ca8f0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:22.674877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 64/138] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:22 2022] PUT /identity/v3/projects/5849148002994ca99651e7026aa6c431/users/4dbfa2988ba9444197b16560d6e65096/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:22.774738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80684bb9-378d-417c-9c9f-d8682fddff91 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:22.815895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10558ef4-d754-49a9-a82f-27fc25aa9f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:22.816917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10558ef4-d754-49a9-a82f-27fc25aa9f61 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.817431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10558ef4-d754-49a9-a82f-27fc25aa9f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.817880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10558ef4-d754-49a9-a82f-27fc25aa9f61 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/users/10d738bcb5484a248894a50e95518057/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.820140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10558ef4-d754-49a9-a82f-27fc25aa9f61 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bf2c16b2e8b435bbd4cf6885ae631db, user_id=10d738bcb5484a248894a50e95518057, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:22.822458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10558ef4-d754-49a9-a82f-27fc25aa9f61 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:22.852397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 75/139] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:22 2022] PUT /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/users/10d738bcb5484a248894a50e95518057/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:22.882534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf06354-9bc0-4be2-ac77-8864362ca8f0 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:22.987636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:23.015157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf06354-9bc0-4be2-ac77-8864362ca8f0 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:23.044607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd914f2-36a1-4da9-9108-f4d6721fa4c4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.044961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd914f2-36a1-4da9-9108-f4d6721fa4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.045169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd914f2-36a1-4da9-9108-f4d6721fa4c4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.046672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 76/140] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:23.052182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf8d2-a532-47bb-9970-fe1fb2b5f745 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.052471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf8d2-a532-47bb-9970-fe1fb2b5f745 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.052776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf8d2-a532-47bb-9970-fe1fb2b5f745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.079006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55363e5-9abe-4371-92d2-33b578969dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:23.079671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55363e5-9abe-4371-92d2-33b578969dda None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.079954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55363e5-9abe-4371-92d2-33b578969dda None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.080238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55363e5-9abe-4371-92d2-33b578969dda None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.081970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55363e5-9abe-4371-92d2-33b578969dda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:23.083530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55363e5-9abe-4371-92d2-33b578969dda None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:23.085228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 65/141] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:23.090257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7cfaf8d2-a532-47bb-9970-fe1fb2b5f745 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:23.091981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55363e5-9abe-4371-92d2-33b578969dda None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:23.095402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7cfaf8d2-a532-47bb-9970-fe1fb2b5f745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:23.191447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:23.197304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 77/142] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:23.227644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55363e5-9abe-4371-92d2-33b578969dda None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:23.263572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2d9f0-e063-4764-8ac9-16f960b16440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:23.264289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2d9f0-e063-4764-8ac9-16f960b16440 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.264582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2d9f0-e063-4764-8ac9-16f960b16440 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.264858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2d9f0-e063-4764-8ac9-16f960b16440 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.265513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2d9f0-e063-4764-8ac9-16f960b16440 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:23.266997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2d9f0-e063-4764-8ac9-16f960b16440 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:23.293540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:23.299056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 66/143] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:23 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:23.587871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaf8d2-a532-47bb-9970-fe1fb2b5f745 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:23.698555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaf8d2-a532-47bb-9970-fe1fb2b5f745 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:23.734430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aab55-b5ff-4a7b-9895-603f60a88909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:23.735189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aab55-b5ff-4a7b-9895-603f60a88909 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.735472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aab55-b5ff-4a7b-9895-603f60a88909 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.735744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aab55-b5ff-4a7b-9895-603f60a88909 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.750079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647aab55-b5ff-4a7b-9895-603f60a88909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-647aab55-b5ff-4a7b-9895-603f60a88909 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:23.755022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:23.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:23.756665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 78/144] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:23.763656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2d9f0-e063-4764-8ac9-16f960b16440 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:23.922810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2d9f0-e063-4764-8ac9-16f960b16440 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:23.967362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:23.967901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.968181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.968447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.969107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:23.969405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:23.970767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:23.982710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 67/145] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:23.988601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aab55-b5ff-4a7b-9895-603f60a88909 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.078510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:24.138097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aab55-b5ff-4a7b-9895-603f60a88909 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.177594 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c758b-2b56-44aa-abf1-76a85fb5e00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.178236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c758b-2b56-44aa-abf1-76a85fb5e00b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.178517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c758b-2b56-44aa-abf1-76a85fb5e00b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.178795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c758b-2b56-44aa-abf1-76a85fb5e00b None admin] PATH_INFO: `/v3/users/a6c35083aa25492e984904fd3fb9f394` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.180011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c758b-2b56-44aa-abf1-76a85fb5e00b None admin] RBAC: Authorizing `identity:get_user(user_id=a6c35083aa25492e984904fd3fb9f394)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.181457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c758b-2b56-44aa-abf1-76a85fb5e00b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.184332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 79/146] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:23 2022] GET /identity/v3/users/a6c35083aa25492e984904fd3fb9f394 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:24.188957 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.259167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:24.313661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d276b25-ef61-4a49-b99b-49475d0e669b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.351691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a889758-4aad-471d-8978-ee4bdd37e5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.352864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889758-4aad-471d-8978-ee4bdd37e5bb None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.353145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889758-4aad-471d-8978-ee4bdd37e5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.353418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889758-4aad-471d-8978-ee4bdd37e5bb None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.357307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a889758-4aad-471d-8978-ee4bdd37e5bb None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.358768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a889758-4aad-471d-8978-ee4bdd37e5bb None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.360462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 68/147] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:24 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:24.366374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c758b-2b56-44aa-abf1-76a85fb5e00b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.471606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c758b-2b56-44aa-abf1-76a85fb5e00b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.498633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:24.507898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cb8eb-3ba9-4595-a5cf-82f141055945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.508776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cb8eb-3ba9-4595-a5cf-82f141055945 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.509338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cb8eb-3ba9-4595-a5cf-82f141055945 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.509563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cb8eb-3ba9-4595-a5cf-82f141055945 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/users/a6c35083aa25492e984904fd3fb9f394/roles/8aad1777fec94e0b8c3dc239ae4864c6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.511573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cb8eb-3ba9-4595-a5cf-82f141055945 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bf2c16b2e8b435bbd4cf6885ae631db, user_id=a6c35083aa25492e984904fd3fb9f394, role_id=8aad1777fec94e0b8c3dc239ae4864c6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.513037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cb8eb-3ba9-4595-a5cf-82f141055945 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.533669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 80/148] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:24 2022] PUT /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/users/a6c35083aa25492e984904fd3fb9f394/roles/8aad1777fec94e0b8c3dc239ae4864c6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:25.699839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18288c8a-9533-4c9d-b3b9-b0ba01e1db7b None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.700330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18288c8a-9533-4c9d-b3b9-b0ba01e1db7b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.700330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18288c8a-9533-4c9d-b3b9-b0ba01e1db7b None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.701792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 69/149] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:25.706852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feda47-981d-4bab-968f-66e43d74bd92 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.707285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feda47-981d-4bab-968f-66e43d74bd92 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.707468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feda47-981d-4bab-968f-66e43d74bd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.731931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-75feda47-981d-4bab-968f-66e43d74bd92 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:25.733148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-75feda47-981d-4bab-968f-66e43d74bd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:25.821365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 81/150] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:26.162265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35991d-baa8-4c39-9d28-c13db95e5616 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:26.162651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35991d-baa8-4c39-9d28-c13db95e5616 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:26.162844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35991d-baa8-4c39-9d28-c13db95e5616 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:26.165018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 70/151] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:26.170369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ebaac-3142-415a-bd56-f1316380a6e7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:26.170662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ebaac-3142-415a-bd56-f1316380a6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:26.170964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ebaac-3142-415a-bd56-f1316380a6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:26.196798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-873ebaac-3142-415a-bd56-f1316380a6e7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:26.198052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-873ebaac-3142-415a-bd56-f1316380a6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:26.288918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 82/152] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:26.360170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017f928-9245-423b-b112-2e6652f02a4f None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:26.360484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017f928-9245-423b-b112-2e6652f02a4f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:26.360733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017f928-9245-423b-b112-2e6652f02a4f None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:26.362210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 71/153] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:26.368654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5182cb-37eb-49b6-ba16-55080b276cd2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:26.368945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5182cb-37eb-49b6-ba16-55080b276cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:26.369226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5182cb-37eb-49b6-ba16-55080b276cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:26.388518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2c5182cb-37eb-49b6-ba16-55080b276cd2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:26.389720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2c5182cb-37eb-49b6-ba16-55080b276cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:26.438550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:26.450380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017f928-9245-423b-b112-2e6652f02a4f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:26.470380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 83/154] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:26.599630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017f928-9245-423b-b112-2e6652f02a4f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:26.631197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:26.638095 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e37473-af24-40f0-ad38-92e8be5469cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:26.639061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e37473-af24-40f0-ad38-92e8be5469cd None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:26.639556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e37473-af24-40f0-ad38-92e8be5469cd None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:26.640029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e37473-af24-40f0-ad38-92e8be5469cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:26.650508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e37473-af24-40f0-ad38-92e8be5469cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-16e37473-af24-40f0-ad38-92e8be5469cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:26.655639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:26.656477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:26.657218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:26.659762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 72/155] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:26.665788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5182cb-37eb-49b6-ba16-55080b276cd2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:26.703096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365faf-7b3c-4b46-907f-0fd046ea1930 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:26.704012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365faf-7b3c-4b46-907f-0fd046ea1930 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:26.704506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365faf-7b3c-4b46-907f-0fd046ea1930 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:26.706062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 73/156] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:26.711766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b6cbf-c488-4e04-b73c-3f1f42a06a42 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:26.712285 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b6cbf-c488-4e04-b73c-3f1f42a06a42 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:26.712767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b6cbf-c488-4e04-b73c-3f1f42a06a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:26.738555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-621b6cbf-c488-4e04-b73c-3f1f42a06a42 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:26.740104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-621b6cbf-c488-4e04-b73c-3f1f42a06a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:26.856018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 74/157] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:26.861781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5182cb-37eb-49b6-ba16-55080b276cd2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:26.924475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:26.925099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:26.925378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:26.925644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:26.926312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:26.926609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:26.928009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:26.942378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 84/158] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:26.948610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b6cbf-c488-4e04-b73c-3f1f42a06a42 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.006950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.083300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:27.085915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b6cbf-c488-4e04-b73c-3f1f42a06a42 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:27.110012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:27.126261 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cf267-68d2-4590-915f-9a6ff5e870d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:27.127064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cf267-68d2-4590-915f-9a6ff5e870d4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.127535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cf267-68d2-4590-915f-9a6ff5e870d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.127884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cf267-68d2-4590-915f-9a6ff5e870d4 None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.129073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8cf267-68d2-4590-915f-9a6ff5e870d4 None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:27.130631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8cf267-68d2-4590-915f-9a6ff5e870d4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:27.133534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 75/159] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:26 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:27.134749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cf267-68d2-4590-915f-9a6ff5e870d4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.226157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a7209-8cbc-49dc-90f9-c62e6c6d1185 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:27.240007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cf267-68d2-4590-915f-9a6ff5e870d4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:27.275555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c671-dfa7-42c6-8ae8-c26835f484bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:27.276269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c671-dfa7-42c6-8ae8-c26835f484bf None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.276622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c671-dfa7-42c6-8ae8-c26835f484bf None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.277006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c671-dfa7-42c6-8ae8-c26835f484bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.287869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991c671-dfa7-42c6-8ae8-c26835f484bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-7991c671-dfa7-42c6-8ae8-c26835f484bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:27.293568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:27.294497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:27.295305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 76/160] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:27.295924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c671-dfa7-42c6-8ae8-c26835f484bf None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.295992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33348cf-63a0-4439-9003-5a716a2b97b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:27.296406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33348cf-63a0-4439-9003-5a716a2b97b1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.298915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33348cf-63a0-4439-9003-5a716a2b97b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.298915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33348cf-63a0-4439-9003-5a716a2b97b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.312565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33348cf-63a0-4439-9003-5a716a2b97b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b33348cf-63a0-4439-9003-5a716a2b97b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:27.318202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:27.319157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:27.319914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:27.335472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 85/161] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:27.336987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33348cf-63a0-4439-9003-5a716a2b97b1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.363244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:27.417769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c671-dfa7-42c6-8ae8-c26835f484bf None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:27.453035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704db13a-a479-4a79-9c36-efa7d5aae401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:27.453791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704db13a-a479-4a79-9c36-efa7d5aae401 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.454192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704db13a-a479-4a79-9c36-efa7d5aae401 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.454546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704db13a-a479-4a79-9c36-efa7d5aae401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.538561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704db13a-a479-4a79-9c36-efa7d5aae401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:27.543223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704db13a-a479-4a79-9c36-efa7d5aae401 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:27.556512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:27.560487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33348cf-63a0-4439-9003-5a716a2b97b1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:27.625553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:27.626189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.626473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.626748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.627441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:27.627745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:27.628248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:27.630357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:27.642249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 86/162] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:27.643110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.705675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 77/163] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:27.706531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704db13a-a479-4a79-9c36-efa7d5aae401 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.822078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167b222-08e7-44f6-b22d-b97b2100f2b7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:27.872183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:27.872796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.873068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.873326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.873993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:27.874295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:27.875735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:27.894110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 87/164] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:27.895325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.941806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:27.952369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704db13a-a479-4a79-9c36-efa7d5aae401 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.029574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de037812-bf77-49d4-9143-88cc0c0f4861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.030225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de037812-bf77-49d4-9143-88cc0c0f4861 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.030508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de037812-bf77-49d4-9143-88cc0c0f4861 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.030779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de037812-bf77-49d4-9143-88cc0c0f4861 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.053609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de037812-bf77-49d4-9143-88cc0c0f4861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-de037812-bf77-49d4-9143-88cc0c0f4861 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:28.058495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:28.063651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:28.064429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:28.065930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 78/165] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:28.066922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de037812-bf77-49d4-9143-88cc0c0f4861 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.086021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.095053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144afd6-9d72-45c2-886f-d3abdccce1d4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.157692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c12fa2-18fe-415d-aa9a-97b1e2c524a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.158684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c12fa2-18fe-415d-aa9a-97b1e2c524a3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.159280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c12fa2-18fe-415d-aa9a-97b1e2c524a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.159860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c12fa2-18fe-415d-aa9a-97b1e2c524a3 None admin] PATH_INFO: `/v3/users/10d738bcb5484a248894a50e95518057` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.166351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c12fa2-18fe-415d-aa9a-97b1e2c524a3 None admin] RBAC: Authorizing `identity:get_user(user_id=10d738bcb5484a248894a50e95518057)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.166351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c12fa2-18fe-415d-aa9a-97b1e2c524a3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.166351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 88/166] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/users/10d738bcb5484a248894a50e95518057 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.167239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c12fa2-18fe-415d-aa9a-97b1e2c524a3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.257822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de037812-bf77-49d4-9143-88cc0c0f4861 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.299245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.312369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c12fa2-18fe-415d-aa9a-97b1e2c524a3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.330869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fec70-834b-47ff-b044-ad39f0f66b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.331991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fec70-834b-47ff-b044-ad39f0f66b1c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.332487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fec70-834b-47ff-b044-ad39f0f66b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.332972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fec70-834b-47ff-b044-ad39f0f66b1c None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.335058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fec70-834b-47ff-b044-ad39f0f66b1c None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.337317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fec70-834b-47ff-b044-ad39f0f66b1c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.341799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 79/167] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.342864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fec70-834b-47ff-b044-ad39f0f66b1c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.385486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ae0f0-8d0e-47fe-af2f-93c5fe22afd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.386528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ae0f0-8d0e-47fe-af2f-93c5fe22afd7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.387059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ae0f0-8d0e-47fe-af2f-93c5fe22afd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.387548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ae0f0-8d0e-47fe-af2f-93c5fe22afd7 None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.390517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ae0f0-8d0e-47fe-af2f-93c5fe22afd7 None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.392515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ae0f0-8d0e-47fe-af2f-93c5fe22afd7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.396758 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 89/168] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.405363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ae0f0-8d0e-47fe-af2f-93c5fe22afd7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.513379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.538940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fec70-834b-47ff-b044-ad39f0f66b1c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.595369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4eb6b-0756-4723-bc4c-e9ce10b20312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.595902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4eb6b-0756-4723-bc4c-e9ce10b20312 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.596180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4eb6b-0756-4723-bc4c-e9ce10b20312 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.596448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4eb6b-0756-4723-bc4c-e9ce10b20312 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.597115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4eb6b-0756-4723-bc4c-e9ce10b20312 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:28.597413 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4eb6b-0756-4723-bc4c-e9ce10b20312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.598800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4eb6b-0756-4723-bc4c-e9ce10b20312 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.605805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ae0f0-8d0e-47fe-af2f-93c5fe22afd7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.609726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 80/169] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.610769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662776b9-b3ee-410b-94de-410d6d84e588 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.611085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662776b9-b3ee-410b-94de-410d6d84e588 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.611359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662776b9-b3ee-410b-94de-410d6d84e588 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.615026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 81/170] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:28.615026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662776b9-b3ee-410b-94de-410d6d84e588 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.652132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffaf4d-c57b-467e-8281-8c6221f831e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.653033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffaf4d-c57b-467e-8281-8c6221f831e9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.653501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffaf4d-c57b-467e-8281-8c6221f831e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.653941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffaf4d-c57b-467e-8281-8c6221f831e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.684099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.738691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffaf4d-c57b-467e-8281-8c6221f831e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.741381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffaf4d-c57b-467e-8281-8c6221f831e9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.760785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662776b9-b3ee-410b-94de-410d6d84e588 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.796621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.797449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.797872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.798315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.799219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] RBAC: Adding query filter params (role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db, user.id=4dbfa2988ba9444197b16560d6e65096) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:28.799667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.801244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.808826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 82/171] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/role_assignments?user.id=4dbfa2988ba9444197b16560d6e65096&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.809894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.821415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.864462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 90/172] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:28.870317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aafb8c-9aeb-4009-a808-2b79d220eac8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.870647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aafb8c-9aeb-4009-a808-2b79d220eac8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.870922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aafb8c-9aeb-4009-a808-2b79d220eac8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.873474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 91/173] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:28.874194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aafb8c-9aeb-4009-a808-2b79d220eac8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.936903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1921a63-0337-4184-b334-33e25fc1715b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.950562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.975307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f9226-c3cc-46fa-a5bb-bc459bc70aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.975927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f9226-c3cc-46fa-a5bb-bc459bc70aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.976214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f9226-c3cc-46fa-a5bb-bc459bc70aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.976486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f9226-c3cc-46fa-a5bb-bc459bc70aa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.008445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aafb8c-9aeb-4009-a808-2b79d220eac8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:29.047994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefee91-3796-4e8d-ab37-d33c7ea31656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:29.048633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefee91-3796-4e8d-ab37-d33c7ea31656 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.048910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefee91-3796-4e8d-ab37-d33c7ea31656 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.049183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefee91-3796-4e8d-ab37-d33c7ea31656 None admin] PATH_INFO: `/v3/users/4dbfa2988ba9444197b16560d6e65096` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.052475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefee91-3796-4e8d-ab37-d33c7ea31656 None admin] RBAC: Authorizing `identity:get_user(user_id=4dbfa2988ba9444197b16560d6e65096)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:29.053903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefee91-3796-4e8d-ab37-d33c7ea31656 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:29.056224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 92/174] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/users/4dbfa2988ba9444197b16560d6e65096 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:29.057175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def27bd-cef2-4cbd-aaa4-baea277d98ab None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.058689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def27bd-cef2-4cbd-aaa4-baea277d98ab None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.058799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def27bd-cef2-4cbd-aaa4-baea277d98ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.059970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f9226-c3cc-46fa-a5bb-bc459bc70aa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:29.060745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f9226-c3cc-46fa-a5bb-bc459bc70aa2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:29.062743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:29.078163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2def27bd-cef2-4cbd-aaa4-baea277d98ab None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:29.079370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2def27bd-cef2-4cbd-aaa4-baea277d98ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:29.145460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 93/175] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:29.146619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103873f7-b31a-4076-8624-aa2170201d21 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.146936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103873f7-b31a-4076-8624-aa2170201d21 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.147212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103873f7-b31a-4076-8624-aa2170201d21 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.151054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 94/176] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:29.151054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d582-76ce-4eb1-b1aa-0d055bc85287 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.151054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d582-76ce-4eb1-b1aa-0d055bc85287 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.151054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d582-76ce-4eb1-b1aa-0d055bc85287 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.155025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 95/177] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:29.155893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674d582-76ce-4eb1-b1aa-0d055bc85287 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:29.162725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 83/178] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:29.163978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59e95d-9e59-4e41-a065-d513b9d65bd4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.164310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59e95d-9e59-4e41-a065-d513b9d65bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.164582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59e95d-9e59-4e41-a065-d513b9d65bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.183704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-8a59e95d-9e59-4e41-a065-d513b9d65bd4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:29.184883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8a59e95d-9e59-4e41-a065-d513b9d65bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:29.223245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:29.252350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 84/179] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:29.253461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59e95d-9e59-4e41-a065-d513b9d65bd4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:29.277013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674d582-76ce-4eb1-b1aa-0d055bc85287 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:29.293303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:29.318269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce07e83-cfa4-4f3d-9505-249a2df756b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:29.319348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce07e83-cfa4-4f3d-9505-249a2df756b7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.319834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce07e83-cfa4-4f3d-9505-249a2df756b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.320296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce07e83-cfa4-4f3d-9505-249a2df756b7 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.321636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce07e83-cfa4-4f3d-9505-249a2df756b7 None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:29.323352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce07e83-cfa4-4f3d-9505-249a2df756b7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:29.326131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 96/180] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:29 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:29.327477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf70de0-2072-4ebf-843f-9b09c6d70afa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.327951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf70de0-2072-4ebf-843f-9b09c6d70afa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.328407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf70de0-2072-4ebf-843f-9b09c6d70afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.350106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-eaf70de0-2072-4ebf-843f-9b09c6d70afa None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:29.351594 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eaf70de0-2072-4ebf-843f-9b09c6d70afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:29.390482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59e95d-9e59-4e41-a065-d513b9d65bd4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:29.442931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 97/181] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:29.442931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98a04-d772-47cd-8a1e-89fb02db28ca None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.442931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98a04-d772-47cd-8a1e-89fb02db28ca None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.442931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98a04-d772-47cd-8a1e-89fb02db28ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.451288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f06ca4-33c4-4632-a187-90c4f90f8811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:29.451903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f06ca4-33c4-4632-a187-90c4f90f8811 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.452178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f06ca4-33c4-4632-a187-90c4f90f8811 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.452445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f06ca4-33c4-4632-a187-90c4f90f8811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.458624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-58e98a04-d772-47cd-8a1e-89fb02db28ca None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:29.459827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-58e98a04-d772-47cd-8a1e-89fb02db28ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:29.512992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:29.531625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 98/182] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:29.532460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e98a04-d772-47cd-8a1e-89fb02db28ca None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:29.537821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f06ca4-33c4-4632-a187-90c4f90f8811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:29.540045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f06ca4-33c4-4632-a187-90c4f90f8811 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:29.579247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:29.652426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e98a04-d772-47cd-8a1e-89fb02db28ca None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:29.661025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 85/183] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:29.661925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f06ca4-33c4-4632-a187-90c4f90f8811 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:29.690578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c20c5-bd2a-4054-a522-1e23db93ae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:29.691733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c20c5-bd2a-4054-a522-1e23db93ae4f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.692208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c20c5-bd2a-4054-a522-1e23db93ae4f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.692697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c20c5-bd2a-4054-a522-1e23db93ae4f None admin] PATH_INFO: `/v3/projects/5849148002994ca99651e7026aa6c431` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.693977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c20c5-bd2a-4054-a522-1e23db93ae4f None admin] RBAC: Authorizing `identity:get_project(project_id=5849148002994ca99651e7026aa6c431)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:29.696465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c20c5-bd2a-4054-a522-1e23db93ae4f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:29.698370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 99/184] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:29 2022] GET /identity/v3/projects/5849148002994ca99651e7026aa6c431 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:29.699549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c20c5-bd2a-4054-a522-1e23db93ae4f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:29.762832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:29.781122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f06ca4-33c4-4632-a187-90c4f90f8811 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:29.813981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c20c5-bd2a-4054-a522-1e23db93ae4f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:29.814157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:29.819559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:29.820166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.821090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.821369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.822094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db, user.id=10d738bcb5484a248894a50e95518057) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:29.822402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:29.823881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:29.829774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 86/185] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:29 2022] GET /identity/v3/role_assignments?user.id=10d738bcb5484a248894a50e95518057&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:29.830681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:29.849828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c798603-0f3c-4e56-944c-b31ed26268a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:29.850484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c798603-0f3c-4e56-944c-b31ed26268a0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.850767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c798603-0f3c-4e56-944c-b31ed26268a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.851086 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c798603-0f3c-4e56-944c-b31ed26268a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:29.863115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c798603-0f3c-4e56-944c-b31ed26268a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4c798603-0f3c-4e56-944c-b31ed26268a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:29.868684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:29.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 100/186] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:29.870970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c798603-0f3c-4e56-944c-b31ed26268a0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:29.950752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e4cf5-adb4-4e49-8c85-d81ae88be333 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:29.987515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4362db-9c41-4175-b46b-0cdc287468e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:29.988166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4362db-9c41-4175-b46b-0cdc287468e7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:29.988270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c798603-0f3c-4e56-944c-b31ed26268a0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:29.988701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4362db-9c41-4175-b46b-0cdc287468e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:29.988701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4362db-9c41-4175-b46b-0cdc287468e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.000336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4362db-9c41-4175-b46b-0cdc287468e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:30.003395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-5b4362db-9c41-4175-b46b-0cdc287468e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:30.005710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:30.007293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:30.008848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 87/187] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:30.009934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4362db-9c41-4175-b46b-0cdc287468e7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.035273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567e876-2a5b-4b88-81ca-67dc81ad3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.035913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e876-2a5b-4b88-81ca-67dc81ad3adf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.036196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e876-2a5b-4b88-81ca-67dc81ad3adf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.036467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e876-2a5b-4b88-81ca-67dc81ad3adf None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.037601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567e876-2a5b-4b88-81ca-67dc81ad3adf None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.039145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567e876-2a5b-4b88-81ca-67dc81ad3adf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.041324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 101/188] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:29 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.042184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567e876-2a5b-4b88-81ca-67dc81ad3adf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.077675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:30.125462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4362db-9c41-4175-b46b-0cdc287468e7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.150020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567e876-2a5b-4b88-81ca-67dc81ad3adf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.158590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.159221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.159501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.159773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.160493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] RBAC: Adding query filter params (role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=5849148002994ca99651e7026aa6c431, user.id=4dbfa2988ba9444197b16560d6e65096) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:30.160797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.162250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.168035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 88/189] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/role_assignments?user.id=4dbfa2988ba9444197b16560d6e65096&scope.project.id=5849148002994ca99651e7026aa6c431&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.168664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.199455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488fc3f-ddb0-44a2-9acb-9cfec7f3f732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.200224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488fc3f-ddb0-44a2-9acb-9cfec7f3f732 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.200573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488fc3f-ddb0-44a2-9acb-9cfec7f3f732 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.200987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488fc3f-ddb0-44a2-9acb-9cfec7f3f732 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.202185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488fc3f-ddb0-44a2-9acb-9cfec7f3f732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.203764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488fc3f-ddb0-44a2-9acb-9cfec7f3f732 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.205513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 102/190] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.206222 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488fc3f-ddb0-44a2-9acb-9cfec7f3f732 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.264031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:30.290493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23cf8f-4acd-4fa6-8f64-66cd6eba1fa6 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.311463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:30.325310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488fc3f-ddb0-44a2-9acb-9cfec7f3f732 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.327095 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f296f-e54c-4ab4-ae9f-6afac41d83cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.327820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f296f-e54c-4ab4-ae9f-6afac41d83cb None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.328176 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f296f-e54c-4ab4-ae9f-6afac41d83cb None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.328551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f296f-e54c-4ab4-ae9f-6afac41d83cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.340848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f296f-e54c-4ab4-ae9f-6afac41d83cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-ac9f296f-e54c-4ab4-ae9f-6afac41d83cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:30.346133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:30.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 89/191] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:30.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f296f-e54c-4ab4-ae9f-6afac41d83cb None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.371141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.371901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.372292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.372660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.373523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:30.373823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.375348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.386797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 103/192] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.386797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.464849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f296f-e54c-4ab4-ae9f-6afac41d83cb None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.495286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f924e-be65-4bc3-b158-4151372fac61 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.497880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.498486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.498767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.499072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.499734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:30.500038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.501408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.505270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:30.509497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 90/193] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.510350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.533480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.534082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.534362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.534634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.535392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] RBAC: Adding query filter params (role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db, user.id=4dbfa2988ba9444197b16560d6e65096) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:30.535700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.537123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.543310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 104/194] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/role_assignments?user.id=4dbfa2988ba9444197b16560d6e65096&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=2e40648646094078acac22f8eb94dd98 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.544124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.554234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:30.618364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba07f-83dc-43ce-9b1c-0d28d458a9f7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.656105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1713f6-edb8-49bb-9ded-cb6dd7ff7857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.657098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1713f6-edb8-49bb-9ded-cb6dd7ff7857 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.657565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1713f6-edb8-49bb-9ded-cb6dd7ff7857 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.657840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1713f6-edb8-49bb-9ded-cb6dd7ff7857 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.658500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1713f6-edb8-49bb-9ded-cb6dd7ff7857 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.660001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1713f6-edb8-49bb-9ded-cb6dd7ff7857 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.663232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960e917-20d0-48cd-a074-a413d186d45d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.683096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 91/195] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:30 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:30.683972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1713f6-edb8-49bb-9ded-cb6dd7ff7857 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.719776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.721063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.721973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.722462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.723442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:30.723956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.726302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.737827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 105/196] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.738834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.781563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:30.837586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:30.845475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1713f6-edb8-49bb-9ded-cb6dd7ff7857 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.879245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0c9a8-a046-437d-be36-69bb877b3096 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:30.890011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c0f96-43df-4723-8a52-12769eedf67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.890771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c0f96-43df-4723-8a52-12769eedf67a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.891192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c0f96-43df-4723-8a52-12769eedf67a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.891566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c0f96-43df-4723-8a52-12769eedf67a None admin] PATH_INFO: `/v3/users/993f2ed94d2f4737b3477ca76762e011` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.906404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c0f96-43df-4723-8a52-12769eedf67a None admin] RBAC: Authorizing `identity:get_user(user_id=993f2ed94d2f4737b3477ca76762e011)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.908040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c0f96-43df-4723-8a52-12769eedf67a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.909768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 92/197] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/users/993f2ed94d2f4737b3477ca76762e011 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.911718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c0f96-43df-4723-8a52-12769eedf67a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:30.919210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877b7b-a20e-42c0-8984-f1f6dc242278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:30.919844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877b7b-a20e-42c0-8984-f1f6dc242278 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:30.920125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877b7b-a20e-42c0-8984-f1f6dc242278 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:30.920398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877b7b-a20e-42c0-8984-f1f6dc242278 None admin] PATH_INFO: `/v3/users/a6c35083aa25492e984904fd3fb9f394` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:30.921539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3877b7b-a20e-42c0-8984-f1f6dc242278 None admin] RBAC: Authorizing `identity:get_user(user_id=a6c35083aa25492e984904fd3fb9f394)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:30.923002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3877b7b-a20e-42c0-8984-f1f6dc242278 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:30.924718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 106/198] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/users/a6c35083aa25492e984904fd3fb9f394 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:30.925336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877b7b-a20e-42c0-8984-f1f6dc242278 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.051113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c0f96-43df-4723-8a52-12769eedf67a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.056710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877b7b-a20e-42c0-8984-f1f6dc242278 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.059470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.095488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b2fde-311b-4892-844c-585e7209f8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.096138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b2fde-311b-4892-844c-585e7209f8ec None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.096424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b2fde-311b-4892-844c-585e7209f8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.096710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b2fde-311b-4892-844c-585e7209f8ec None admin] PATH_INFO: `/v3/users/993f2ed94d2f4737b3477ca76762e011` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.097829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b2fde-311b-4892-844c-585e7209f8ec None admin] RBAC: Authorizing `identity:get_user(user_id=993f2ed94d2f4737b3477ca76762e011)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.099354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b2fde-311b-4892-844c-585e7209f8ec None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.099962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea3974-ddb2-4868-9668-e83c108f5dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.100729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3974-ddb2-4868-9668-e83c108f5dad None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.100941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 93/199] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/users/993f2ed94d2f4737b3477ca76762e011 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:31.101286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3974-ddb2-4868-9668-e83c108f5dad None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.102238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3974-ddb2-4868-9668-e83c108f5dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.102924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b2fde-311b-4892-844c-585e7209f8ec None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.139178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.178400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea3974-ddb2-4868-9668-e83c108f5dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.180919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea3974-ddb2-4868-9668-e83c108f5dad None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.237290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b2fde-311b-4892-844c-585e7209f8ec None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.286616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682051b-55d8-489a-8012-21a4e6b54b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.288048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682051b-55d8-489a-8012-21a4e6b54b2e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.288476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682051b-55d8-489a-8012-21a4e6b54b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.288889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682051b-55d8-489a-8012-21a4e6b54b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.291231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 107/200] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:31.292213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea3974-ddb2-4868-9668-e83c108f5dad None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.295503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.372812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682051b-55d8-489a-8012-21a4e6b54b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.374561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682051b-55d8-489a-8012-21a4e6b54b2e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.421647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.426187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea3974-ddb2-4868-9668-e83c108f5dad None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.464449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c5793-a80b-4bbc-b050-cf7c635e1315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.465015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c5793-a80b-4bbc-b050-cf7c635e1315 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.465496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c5793-a80b-4bbc-b050-cf7c635e1315 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.465795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c5793-a80b-4bbc-b050-cf7c635e1315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.509673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 94/201] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:31.510770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682051b-55d8-489a-8012-21a4e6b54b2e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.538456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c5793-a80b-4bbc-b050-cf7c635e1315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.540269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c5793-a80b-4bbc-b050-cf7c635e1315 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.543070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.619059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682051b-55d8-489a-8012-21a4e6b54b2e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.644474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.651620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 108/202] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:31.653098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c5793-a80b-4bbc-b050-cf7c635e1315 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.657121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2907f4a-ade3-4e99-b710-98be0bd83167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.658015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907f4a-ade3-4e99-b710-98be0bd83167 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.658469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907f4a-ade3-4e99-b710-98be0bd83167 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.658934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2907f4a-ade3-4e99-b710-98be0bd83167 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.660246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2907f4a-ade3-4e99-b710-98be0bd83167 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.661888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2907f4a-ade3-4e99-b710-98be0bd83167 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.663779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 95/203] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:31.665518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2907f4a-ade3-4e99-b710-98be0bd83167 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.761921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c5793-a80b-4bbc-b050-cf7c635e1315 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.764434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.772616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2907f4a-ade3-4e99-b710-98be0bd83167 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.800235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae5aa9-e52d-4728-a6c6-6add606c2f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.800947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae5aa9-e52d-4728-a6c6-6add606c2f86 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.801234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae5aa9-e52d-4728-a6c6-6add606c2f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.801567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae5aa9-e52d-4728-a6c6-6add606c2f86 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.803014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae5aa9-e52d-4728-a6c6-6add606c2f86 None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.804440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae5aa9-e52d-4728-a6c6-6add606c2f86 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.806773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 109/204] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:31.807711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae5aa9-e52d-4728-a6c6-6add606c2f86 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.812325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30037da1-1cdd-4a89-895b-5acdc9cacf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.813352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037da1-1cdd-4a89-895b-5acdc9cacf76 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.813855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037da1-1cdd-4a89-895b-5acdc9cacf76 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.814322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037da1-1cdd-4a89-895b-5acdc9cacf76 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.816258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30037da1-1cdd-4a89-895b-5acdc9cacf76 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.817999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30037da1-1cdd-4a89-895b-5acdc9cacf76 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.820064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 96/205] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:31.821101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30037da1-1cdd-4a89-895b-5acdc9cacf76 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.882917 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.915822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae5aa9-e52d-4728-a6c6-6add606c2f86 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.927766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30037da1-1cdd-4a89-895b-5acdc9cacf76 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.952000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.952631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.952923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.953208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.953941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591, user.id=993f2ed94d2f4737b3477ca76762e011) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:31.954426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.955712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.961500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 110/206] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/role_assignments?user.id=993f2ed94d2f4737b3477ca76762e011&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:31.962331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.967369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609d9e3-21f2-454d-97d1-2966c98d686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.968590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609d9e3-21f2-454d-97d1-2966c98d686e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.969101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609d9e3-21f2-454d-97d1-2966c98d686e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.969596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609d9e3-21f2-454d-97d1-2966c98d686e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.970554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609d9e3-21f2-454d-97d1-2966c98d686e None admin] RBAC: Adding query filter params (role.id=8aad1777fec94e0b8c3dc239ae4864c6, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db, user.id=a6c35083aa25492e984904fd3fb9f394) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:31.971112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609d9e3-21f2-454d-97d1-2966c98d686e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.972804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609d9e3-21f2-454d-97d1-2966c98d686e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.980335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 97/207] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/role_assignments?user.id=a6c35083aa25492e984904fd3fb9f394&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=8aad1777fec94e0b8c3dc239ae4864c6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:32.005543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:32.087860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726a295-be50-45ae-8d02-55e351faaf18 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:32.127084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fc68e-8659-4821-a1b7-0a0085dc6b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:32.127704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fc68e-8659-4821-a1b7-0a0085dc6b49 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:32.127989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fc68e-8659-4821-a1b7-0a0085dc6b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:32.128266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fc68e-8659-4821-a1b7-0a0085dc6b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:32.129007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fc68e-8659-4821-a1b7-0a0085dc6b49 None admin] RBAC: Adding query filter params (role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591, user.id=993f2ed94d2f4737b3477ca76762e011) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:32.129317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fc68e-8659-4821-a1b7-0a0085dc6b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:32.130798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fc68e-8659-4821-a1b7-0a0085dc6b49 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:32.138230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 111/208] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/role_assignments?user.id=993f2ed94d2f4737b3477ca76762e011&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=2e40648646094078acac22f8eb94dd98 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:33.252232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf52212-8b9e-42fd-99e6-2df2ecc4b0f7 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:33.252744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf52212-8b9e-42fd-99e6-2df2ecc4b0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:33.252818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf52212-8b9e-42fd-99e6-2df2ecc4b0f7 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:33.254141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 98/209] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:33.259937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28b5f-e1c6-4081-8949-7af20563b09e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:33.260332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28b5f-e1c6-4081-8949-7af20563b09e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:33.260533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28b5f-e1c6-4081-8949-7af20563b09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:33.279338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-afd28b5f-e1c6-4081-8949-7af20563b09e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:33.280504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-afd28b5f-e1c6-4081-8949-7af20563b09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:33.303737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73fd4f-9487-4c98-9198-990edc2c6d02 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:33.304346 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73fd4f-9487-4c98-9198-990edc2c6d02 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:33.304716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73fd4f-9487-4c98-9198-990edc2c6d02 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:33.306250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 99/210] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:33.311358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641b53-30fd-4168-990a-0e413aba0a85 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:33.311730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641b53-30fd-4168-990a-0e413aba0a85 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:33.312082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641b53-30fd-4168-990a-0e413aba0a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:33.333323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ed641b53-30fd-4168-990a-0e413aba0a85 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:33.334613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ed641b53-30fd-4168-990a-0e413aba0a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:33.351530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 112/211] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:33.407722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 100/212] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:33.624454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd28b5f-e1c6-4081-8949-7af20563b09e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:33.654859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:33.684811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed641b53-30fd-4168-990a-0e413aba0a85 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:33.725391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd28b5f-e1c6-4081-8949-7af20563b09e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:33.746058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:33.761199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef717714-9c07-40dd-81d7-6284095e10f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:33.761477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef717714-9c07-40dd-81d7-6284095e10f5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:33.761700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef717714-9c07-40dd-81d7-6284095e10f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:33.761963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef717714-9c07-40dd-81d7-6284095e10f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:33.763220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef717714-9c07-40dd-81d7-6284095e10f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:33.764555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef717714-9c07-40dd-81d7-6284095e10f5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:33.766940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 113/213] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:33.772116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef717714-9c07-40dd-81d7-6284095e10f5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:33.799582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed641b53-30fd-4168-990a-0e413aba0a85 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:33.838110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2c1f-29d9-4535-8e3d-8bcd47121db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:33.839104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2c1f-29d9-4535-8e3d-8bcd47121db0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:33.839582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2c1f-29d9-4535-8e3d-8bcd47121db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:33.840044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2c1f-29d9-4535-8e3d-8bcd47121db0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:33.851537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2c1f-29d9-4535-8e3d-8bcd47121db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-497c2c1f-29d9-4535-8e3d-8bcd47121db0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:33.857089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:33.857933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:33.858729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:33.865927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 101/214] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:33.868053 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2c1f-29d9-4535-8e3d-8bcd47121db0 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:33.872484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:33.876267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef717714-9c07-40dd-81d7-6284095e10f5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:33.912455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb89c3-4697-48e3-874f-702f65e41d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:33.913213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb89c3-4697-48e3-874f-702f65e41d91 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:33.913589 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb89c3-4697-48e3-874f-702f65e41d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:33.914515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb89c3-4697-48e3-874f-702f65e41d91 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:33.915296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb89c3-4697-48e3-874f-702f65e41d91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:33.916874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb89c3-4697-48e3-874f-702f65e41d91 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:33.954994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 114/215] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:33 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:45:33.993130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2c1f-29d9-4535-8e3d-8bcd47121db0 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:34.010524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:34.033865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:34.034494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:34.034840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:34.035142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:34.035822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:34.036124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:34.037547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:34.047999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 102/216] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:34.053678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb89c3-4697-48e3-874f-702f65e41d91 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:34.171097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb89c3-4697-48e3-874f-702f65e41d91 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:34.207541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22087291-c960-44d7-a952-647e9244e0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:34.208503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087291-c960-44d7-a952-647e9244e0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:34.208780 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087291-c960-44d7-a952-647e9244e0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:34.209071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087291-c960-44d7-a952-647e9244e0b8 None admin] PATH_INFO: `/v3/users/69a894572e7742348231959c24b7897c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:34.224471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22087291-c960-44d7-a952-647e9244e0b8 None admin] RBAC: Authorizing `identity:get_user(user_id=69a894572e7742348231959c24b7897c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:34.226089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22087291-c960-44d7-a952-647e9244e0b8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:34.228337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 115/217] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:34 2022] GET /identity/v3/users/69a894572e7742348231959c24b7897c => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:34.234198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:34.349587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3192db-753f-46da-8b4e-62febe8952d2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:34.392408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c24107-32ed-4a23-b7e4-ea818d953bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:34.393034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c24107-32ed-4a23-b7e4-ea818d953bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:34.393324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c24107-32ed-4a23-b7e4-ea818d953bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:34.393588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c24107-32ed-4a23-b7e4-ea818d953bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:34.398674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd5f1-ab65-4626-b5a6-f85cb2166d53 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:34.399614 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd5f1-ab65-4626-b5a6-f85cb2166d53 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:34.400079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd5f1-ab65-4626-b5a6-f85cb2166d53 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:34.401927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 116/218] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:34.410321 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953bae-548e-4c13-a394-170f39c47fad None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:34.411030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953bae-548e-4c13-a394-170f39c47fad None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:34.411496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953bae-548e-4c13-a394-170f39c47fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:34.435014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-17953bae-548e-4c13-a394-170f39c47fad None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:34.436511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-17953bae-548e-4c13-a394-170f39c47fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:34.446782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:34.456901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:34.488846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c24107-32ed-4a23-b7e4-ea818d953bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:34.490650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c24107-32ed-4a23-b7e4-ea818d953bd5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:34.535470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 117/219] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:34.540696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9afa-2041-4dde-a73c-9c5798501493 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:34.541193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9afa-2041-4dde-a73c-9c5798501493 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:34.541705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9afa-2041-4dde-a73c-9c5798501493 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:34.543203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 118/220] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:34.554956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbacb-3470-4f32-87e6-e5323644685a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:34.555532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbacb-3470-4f32-87e6-e5323644685a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:34.556026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbacb-3470-4f32-87e6-e5323644685a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:34.589763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-285cbacb-3470-4f32-87e6-e5323644685a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:34.592060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-285cbacb-3470-4f32-87e6-e5323644685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:34.671515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 103/221] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:34.681393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c24107-32ed-4a23-b7e4-ea818d953bd5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:34.744746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 119/222] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:34.796740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:34.826729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c24107-32ed-4a23-b7e4-ea818d953bd5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:34.876904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28952d17-6a58-4455-956e-d477b9c5e4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:34.877536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28952d17-6a58-4455-956e-d477b9c5e4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:34.877872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28952d17-6a58-4455-956e-d477b9c5e4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:34.878145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28952d17-6a58-4455-956e-d477b9c5e4d4 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:34.879312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28952d17-6a58-4455-956e-d477b9c5e4d4 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:34.880751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28952d17-6a58-4455-956e-d477b9c5e4d4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:34.882361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 104/223] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:34 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:34.886624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cbacb-3470-4f32-87e6-e5323644685a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:34.898083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28952d17-6a58-4455-956e-d477b9c5e4d4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.001408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.014786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28952d17-6a58-4455-956e-d477b9c5e4d4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.064925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.065127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cbacb-3470-4f32-87e6-e5323644685a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.065753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.066067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.066343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.067126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] RBAC: Adding query filter params (role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591, user.id=69a894572e7742348231959c24b7897c) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:35.067441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:35.069197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:35.081715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 105/224] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:34 2022] GET /identity/v3/role_assignments?user.id=69a894572e7742348231959c24b7897c&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:35.085337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.099319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.106024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bebaf-b6b4-4651-afbb-47bd81892195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.106689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bebaf-b6b4-4651-afbb-47bd81892195 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.106992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bebaf-b6b4-4651-afbb-47bd81892195 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.107265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bebaf-b6b4-4651-afbb-47bd81892195 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.119941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bebaf-b6b4-4651-afbb-47bd81892195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6b5bebaf-b6b4-4651-afbb-47bd81892195 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:35.125842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:35.126767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:35.127586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 120/225] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:35.134784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bebaf-b6b4-4651-afbb-47bd81892195 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.215683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea0a93-15c2-446f-adfb-e6529a09cbb3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.258390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a205d8-d050-43fd-ba55-d971a75a7b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.259162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a205d8-d050-43fd-ba55-d971a75a7b85 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.259444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a205d8-d050-43fd-ba55-d971a75a7b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.259775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a205d8-d050-43fd-ba55-d971a75a7b85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.271871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a205d8-d050-43fd-ba55-d971a75a7b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-90a205d8-d050-43fd-ba55-d971a75a7b85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:35.277126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:35.279583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:35.281511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 106/226] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:35.284498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a205d8-d050-43fd-ba55-d971a75a7b85 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.337185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bebaf-b6b4-4651-afbb-47bd81892195 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.362787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.404272 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.409066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.409824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a205d8-d050-43fd-ba55-d971a75a7b85 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.410339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.411172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.411649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.414854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:35.415400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:35.418041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:35.431819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 121/227] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:35.437043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.455415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.456297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.456738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.457134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.457932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:35.458361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:35.459891 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:35.473422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 107/228] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:35.474994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.581836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5986-ddb0-4ee6-a31e-fe4c5a293c85 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.612124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.619905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf212a-a14b-4d1d-85f7-6f6bfa170768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.620548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf212a-a14b-4d1d-85f7-6f6bfa170768 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.620830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf212a-a14b-4d1d-85f7-6f6bfa170768 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.621102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf212a-a14b-4d1d-85f7-6f6bfa170768 None admin] PATH_INFO: `/v3/users/993f2ed94d2f4737b3477ca76762e011` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.622239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf212a-a14b-4d1d-85f7-6f6bfa170768 None admin] RBAC: Authorizing `identity:get_user(user_id=993f2ed94d2f4737b3477ca76762e011)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:35.623851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf212a-a14b-4d1d-85f7-6f6bfa170768 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:35.625443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 108/229] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:35 2022] GET /identity/v3/users/993f2ed94d2f4737b3477ca76762e011 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:35.631135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf212a-a14b-4d1d-85f7-6f6bfa170768 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.631698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b6483-4c6a-4b26-af40-d4f916dc6c32 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.702687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb40b0-f1f2-49e6-8f25-ace2e9ce945d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.703453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb40b0-f1f2-49e6-8f25-ace2e9ce945d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.703739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb40b0-f1f2-49e6-8f25-ace2e9ce945d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.704013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb40b0-f1f2-49e6-8f25-ace2e9ce945d None admin] PATH_INFO: `/v3/users/993f2ed94d2f4737b3477ca76762e011` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.705330 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb40b0-f1f2-49e6-8f25-ace2e9ce945d None admin] RBAC: Authorizing `identity:get_user(user_id=993f2ed94d2f4737b3477ca76762e011)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:35.706708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb40b0-f1f2-49e6-8f25-ace2e9ce945d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:35.711585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 122/230] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:35 2022] GET /identity/v3/users/993f2ed94d2f4737b3477ca76762e011 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:35.717063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb40b0-f1f2-49e6-8f25-ace2e9ce945d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.741807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf212a-a14b-4d1d-85f7-6f6bfa170768 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.785286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8075fe-b34f-4854-8e7b-0d4dd0ac817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.786164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8075fe-b34f-4854-8e7b-0d4dd0ac817c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.787334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8075fe-b34f-4854-8e7b-0d4dd0ac817c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.787837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8075fe-b34f-4854-8e7b-0d4dd0ac817c None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.791163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8075fe-b34f-4854-8e7b-0d4dd0ac817c None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:35.795068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8075fe-b34f-4854-8e7b-0d4dd0ac817c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:35.804122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 109/231] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:35 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:35.806038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8075fe-b34f-4854-8e7b-0d4dd0ac817c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.863338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.888491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.923482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8075fe-b34f-4854-8e7b-0d4dd0ac817c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.962812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9f7a6-78fa-44b4-9afd-7d42670b1c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:35.963507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9f7a6-78fa-44b4-9afd-7d42670b1c54 None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.964076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9f7a6-78fa-44b4-9afd-7d42670b1c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.964407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9f7a6-78fa-44b4-9afd-7d42670b1c54 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/users/993f2ed94d2f4737b3477ca76762e011/roles/2e40648646094078acac22f8eb94dd98` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.966746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9f7a6-78fa-44b4-9afd-7d42670b1c54 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcc729b56f9c4ffb9e49f733ba16d591, user_id=993f2ed94d2f4737b3477ca76762e011, role_id=2e40648646094078acac22f8eb94dd98)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:35.968219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9f7a6-78fa-44b4-9afd-7d42670b1c54 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:35.977209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb40b0-f1f2-49e6-8f25-ace2e9ce945d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:35.984777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 110/232] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:35 2022] PUT /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/users/993f2ed94d2f4737b3477ca76762e011/roles/2e40648646094078acac22f8eb94dd98 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:45:36.021020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3695d4-1569-47b8-85bd-e5986924f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.021728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3695d4-1569-47b8-85bd-e5986924f706 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.022013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3695d4-1569-47b8-85bd-e5986924f706 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.022287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3695d4-1569-47b8-85bd-e5986924f706 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.024236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3695d4-1569-47b8-85bd-e5986924f706 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.025737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3695d4-1569-47b8-85bd-e5986924f706 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.027788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 123/233] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:35 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:36.033338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9f7a6-78fa-44b4-9afd-7d42670b1c54 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.174081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9f7a6-78fa-44b4-9afd-7d42670b1c54 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.202770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.211218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c284850-1e87-4fda-a567-6ccf6ef8b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.212547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284850-1e87-4fda-a567-6ccf6ef8b672 None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.212840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284850-1e87-4fda-a567-6ccf6ef8b672 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.213648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284850-1e87-4fda-a567-6ccf6ef8b672 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/users/993f2ed94d2f4737b3477ca76762e011/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.215722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c284850-1e87-4fda-a567-6ccf6ef8b672 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcc729b56f9c4ffb9e49f733ba16d591, user_id=993f2ed94d2f4737b3477ca76762e011, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.217287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c284850-1e87-4fda-a567-6ccf6ef8b672 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.237617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 111/234] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:36 2022] PUT /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/users/993f2ed94d2f4737b3477ca76762e011/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:36.839807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945686f-c345-480f-8800-45308024f530 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.840319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945686f-c345-480f-8800-45308024f530 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.840422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945686f-c345-480f-8800-45308024f530 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.842002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 124/235] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:36.856738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4d47-3a3c-4c11-b35e-eaa87384b7f7 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.857177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4d47-3a3c-4c11-b35e-eaa87384b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.857358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4d47-3a3c-4c11-b35e-eaa87384b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.884236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-16cd4d47-3a3c-4c11-b35e-eaa87384b7f7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:36.885437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-16cd4d47-3a3c-4c11-b35e-eaa87384b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:36.893496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3596-d859-4db4-9029-bc58e017847b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.893845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3596-d859-4db4-9029-bc58e017847b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.894127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3596-d859-4db4-9029-bc58e017847b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.895506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 125/236] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:36.900526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534dc5d-68e7-4f70-857c-5386bc1f8da6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.900812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534dc5d-68e7-4f70-857c-5386bc1f8da6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.901325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534dc5d-68e7-4f70-857c-5386bc1f8da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.926270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e534dc5d-68e7-4f70-857c-5386bc1f8da6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:36.927497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e534dc5d-68e7-4f70-857c-5386bc1f8da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:37.006058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 126/237] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:37.009396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 112/238] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:37.354089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534dc5d-68e7-4f70-857c-5386bc1f8da6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.423954 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.444358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd4d47-3a3c-4c11-b35e-eaa87384b7f7 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.467057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534dc5d-68e7-4f70-857c-5386bc1f8da6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.502174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88ad71-c610-4898-bc0b-abe3db811575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.503073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88ad71-c610-4898-bc0b-abe3db811575 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.503498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88ad71-c610-4898-bc0b-abe3db811575 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.503940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88ad71-c610-4898-bc0b-abe3db811575 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.505238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88ad71-c610-4898-bc0b-abe3db811575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.506852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88ad71-c610-4898-bc0b-abe3db811575 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.509514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 127/239] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:37.516183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88ad71-c610-4898-bc0b-abe3db811575 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.633869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd4d47-3a3c-4c11-b35e-eaa87384b7f7 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.642877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88ad71-c610-4898-bc0b-abe3db811575 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.676458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.678065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.684381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9eec2-0c0a-4b42-96a2-bc522c52994e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.685439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9eec2-0c0a-4b42-96a2-bc522c52994e None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.686129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9eec2-0c0a-4b42-96a2-bc522c52994e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.686700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9eec2-0c0a-4b42-96a2-bc522c52994e None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.688024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9eec2-0c0a-4b42-96a2-bc522c52994e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.689732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9eec2-0c0a-4b42-96a2-bc522c52994e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.715303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 128/240] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:37 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:37.716924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba622965-0940-4daa-85ce-93ffa6da7add None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.717420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba622965-0940-4daa-85ce-93ffa6da7add None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.717597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c04af4-0fe1-48c3-bf8f-012b57067af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.717999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba622965-0940-4daa-85ce-93ffa6da7add None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.718737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c04af4-0fe1-48c3-bf8f-012b57067af3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.719192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c04af4-0fe1-48c3-bf8f-012b57067af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.719619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c04af4-0fe1-48c3-bf8f-012b57067af3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.719957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 129/241] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:37.729580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e100a-7dde-40ad-95e1-8054a02e845d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.729902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e100a-7dde-40ad-95e1-8054a02e845d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.730132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e100a-7dde-40ad-95e1-8054a02e845d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.734347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c04af4-0fe1-48c3-bf8f-012b57067af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-42c04af4-0fe1-48c3-bf8f-012b57067af3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:37.740059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:37.740998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:37.741776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:37.750050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 113/242] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:37.751831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c04af4-0fe1-48c3-bf8f-012b57067af3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.788064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e77e100a-7dde-40ad-95e1-8054a02e845d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:37.789392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e77e100a-7dde-40ad-95e1-8054a02e845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:37.873193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c04af4-0fe1-48c3-bf8f-012b57067af3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.920740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.921269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.921546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.921817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.922471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:37.922770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.924250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.933405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 114/243] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:37.938763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.949878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 130/244] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:37.980706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.043793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c159720-9e08-4ddc-89eb-f8326eb7d613 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.078524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb03db-731d-4d94-93de-afa6d9bb6797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.079191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb03db-731d-4d94-93de-afa6d9bb6797 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.079466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb03db-731d-4d94-93de-afa6d9bb6797 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.079734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb03db-731d-4d94-93de-afa6d9bb6797 None admin] PATH_INFO: `/v3/users/0edd1a05d8314510975c83747cf0b430` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.093743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb03db-731d-4d94-93de-afa6d9bb6797 None admin] RBAC: Authorizing `identity:get_user(user_id=0edd1a05d8314510975c83747cf0b430)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.095224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb03db-731d-4d94-93de-afa6d9bb6797 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.096851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 115/245] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/users/0edd1a05d8314510975c83747cf0b430 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:38.102874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e100a-7dde-40ad-95e1-8054a02e845d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.228266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e100a-7dde-40ad-95e1-8054a02e845d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.244342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb03db-731d-4d94-93de-afa6d9bb6797 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.245217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.271995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b039-2f8d-4f7d-8e18-cda5c1f9f1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.272962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b039-2f8d-4f7d-8e18-cda5c1f9f1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.273971 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b039-2f8d-4f7d-8e18-cda5c1f9f1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.274497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b039-2f8d-4f7d-8e18-cda5c1f9f1d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.359239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b039-2f8d-4f7d-8e18-cda5c1f9f1d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.360709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b039-2f8d-4f7d-8e18-cda5c1f9f1d9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.371054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.380549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb03db-731d-4d94-93de-afa6d9bb6797 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.427599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a566f-4eab-44d9-9541-f6e0b35a3d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.428236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a566f-4eab-44d9-9541-f6e0b35a3d94 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.428515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a566f-4eab-44d9-9541-f6e0b35a3d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.428787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a566f-4eab-44d9-9541-f6e0b35a3d94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.443019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a566f-4eab-44d9-9541-f6e0b35a3d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-cc3a566f-4eab-44d9-9541-f6e0b35a3d94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:38.448575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:38.449435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:38.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 116/246] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:38.458988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a566f-4eab-44d9-9541-f6e0b35a3d94 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.488165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 131/247] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:38.494721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b039-2f8d-4f7d-8e18-cda5c1f9f1d9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.516923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.597278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a566f-4eab-44d9-9541-f6e0b35a3d94 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.642782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b039-2f8d-4f7d-8e18-cda5c1f9f1d9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.649856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.650796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.651538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.652063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.653585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:38.654099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.656178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.669247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 117/248] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:38.675406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.701864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fcfc3-054e-4854-8d54-58bbcd337893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.702565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fcfc3-054e-4854-8d54-58bbcd337893 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.702851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fcfc3-054e-4854-8d54-58bbcd337893 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.703145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fcfc3-054e-4854-8d54-58bbcd337893 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.707177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fcfc3-054e-4854-8d54-58bbcd337893 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.708697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fcfc3-054e-4854-8d54-58bbcd337893 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.713235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 132/249] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:38.716503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.719076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fcfc3-054e-4854-8d54-58bbcd337893 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.836695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.859274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47743c58-97a4-4f6b-80d0-8e31f37b2a21 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.859469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fcfc3-054e-4854-8d54-58bbcd337893 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.895910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc4892-3a00-46f3-a663-bf508d5108d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.896525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc4892-3a00-46f3-a663-bf508d5108d5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.896807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc4892-3a00-46f3-a663-bf508d5108d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.897085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc4892-3a00-46f3-a663-bf508d5108d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.897820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc4892-3a00-46f3-a663-bf508d5108d5 None admin] RBAC: Adding query filter params (role.id=8aad1777fec94e0b8c3dc239ae4864c6, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591, user.id=0edd1a05d8314510975c83747cf0b430) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:38.898650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc4892-3a00-46f3-a663-bf508d5108d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.900221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc4892-3a00-46f3-a663-bf508d5108d5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.906644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 133/250] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/role_assignments?user.id=0edd1a05d8314510975c83747cf0b430&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=8aad1777fec94e0b8c3dc239ae4864c6 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:38.907903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7c27c-f63d-4948-bdff-eedd0841f601 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.908189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7c27c-f63d-4948-bdff-eedd0841f601 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.908460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7c27c-f63d-4948-bdff-eedd0841f601 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.914866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 134/251] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:38.916637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf86e7c-84dd-4372-a612-245469e35ef3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.916932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf86e7c-84dd-4372-a612-245469e35ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.917211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf86e7c-84dd-4372-a612-245469e35ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.938498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ebf86e7c-84dd-4372-a612-245469e35ef3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:38.940189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ebf86e7c-84dd-4372-a612-245469e35ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:38.959754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c27ead-f881-450c-a92e-fe14db4408b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.960411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c27ead-f881-450c-a92e-fe14db4408b2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.961221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c27ead-f881-450c-a92e-fe14db4408b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.961495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c27ead-f881-450c-a92e-fe14db4408b2 None admin] PATH_INFO: `/v3/users/69a894572e7742348231959c24b7897c` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.967658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c27ead-f881-450c-a92e-fe14db4408b2 None admin] RBAC: Authorizing `identity:get_user(user_id=69a894572e7742348231959c24b7897c)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.969171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c27ead-f881-450c-a92e-fe14db4408b2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.971251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 118/252] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/users/69a894572e7742348231959c24b7897c => generated 297 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:38.977134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c27ead-f881-450c-a92e-fe14db4408b2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:39.018310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 135/253] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:39.144109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:39.154443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c27ead-f881-450c-a92e-fe14db4408b2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:39.190217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fb44d-e090-4ef2-a38a-efc6bfb95f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:39.190850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fb44d-e090-4ef2-a38a-efc6bfb95f01 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.191151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fb44d-e090-4ef2-a38a-efc6bfb95f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.191420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fb44d-e090-4ef2-a38a-efc6bfb95f01 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:39.192544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fb44d-e090-4ef2-a38a-efc6bfb95f01 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:39.193984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fb44d-e090-4ef2-a38a-efc6bfb95f01 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:39.195971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 119/254] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:39.201868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf86e7c-84dd-4372-a612-245469e35ef3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:39.289499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:39.319004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf86e7c-84dd-4372-a612-245469e35ef3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:39.372489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef86ccf-e2f8-4a0d-b2b1-1d06d2548333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:39.373157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86ccf-e2f8-4a0d-b2b1-1d06d2548333 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.373439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86ccf-e2f8-4a0d-b2b1-1d06d2548333 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.373712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86ccf-e2f8-4a0d-b2b1-1d06d2548333 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/users/69a894572e7742348231959c24b7897c/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:39.380220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef86ccf-e2f8-4a0d-b2b1-1d06d2548333 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcc729b56f9c4ffb9e49f733ba16d591, user_id=69a894572e7742348231959c24b7897c, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:39.381663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef86ccf-e2f8-4a0d-b2b1-1d06d2548333 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:39.423822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 136/255] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:39 2022] PUT /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/users/69a894572e7742348231959c24b7897c/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:39.457891 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7b202-5f11-4b78-897c-58085a4328c5 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.458231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7b202-5f11-4b78-897c-58085a4328c5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.458450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7b202-5f11-4b78-897c-58085a4328c5 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:39.459861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 120/256] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:39.467705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeca22-0c2e-4999-a7dc-63d8f26f3888 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.468037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeca22-0c2e-4999-a7dc-63d8f26f3888 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.468279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeca22-0c2e-4999-a7dc-63d8f26f3888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:39.497395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-acdeca22-0c2e-4999-a7dc-63d8f26f3888 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:39.498624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-acdeca22-0c2e-4999-a7dc-63d8f26f3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:39.553670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7b202-5f11-4b78-897c-58085a4328c5 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:39.616810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 137/257] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:39.732475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7b202-5f11-4b78-897c-58085a4328c5 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:39.771582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3f75a-20a3-4c35-a811-16ca60306111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:39.772215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3f75a-20a3-4c35-a811-16ca60306111 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.772493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3f75a-20a3-4c35-a811-16ca60306111 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.772757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3f75a-20a3-4c35-a811-16ca60306111 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:39.783731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3f75a-20a3-4c35-a811-16ca60306111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-64e3f75a-20a3-4c35-a811-16ca60306111 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:39.788494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:39.789956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:39.790732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 121/258] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:39.797584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeca22-0c2e-4999-a7dc-63d8f26f3888 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:39.899526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:39.939535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeca22-0c2e-4999-a7dc-63d8f26f3888 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:39.952189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3f75a-20a3-4c35-a811-16ca60306111 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:39.979265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:40.021001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.025166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.025455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.025733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.027677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:40.028059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:40.029454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:40.058260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 138/259] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:40.072176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.166541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3f75a-20a3-4c35-a811-16ca60306111 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.235220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a4fdb-35be-47e5-9f64-285c846c75ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.236457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4fdb-35be-47e5-9f64-285c846c75ac None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.236967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4fdb-35be-47e5-9f64-285c846c75ac None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.237432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4fdb-35be-47e5-9f64-285c846c75ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.259395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a4fdb-35be-47e5-9f64-285c846c75ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-379a4fdb-35be-47e5-9f64-285c846c75ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:40.264233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:40.265165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:40.265927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 122/260] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:40.272873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a4fdb-35be-47e5-9f64-285c846c75ac None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.307615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fa29d-bfda-4f73-8201-148a42083234 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.356926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:40.371302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22713e-cbb3-4d85-9cfa-6478ea156cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.371942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22713e-cbb3-4d85-9cfa-6478ea156cda None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.372221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22713e-cbb3-4d85-9cfa-6478ea156cda None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.372494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22713e-cbb3-4d85-9cfa-6478ea156cda None admin] PATH_INFO: `/v3/users/993f2ed94d2f4737b3477ca76762e011` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.374273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22713e-cbb3-4d85-9cfa-6478ea156cda None admin] RBAC: Authorizing `identity:get_user(user_id=993f2ed94d2f4737b3477ca76762e011)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:40.377251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22713e-cbb3-4d85-9cfa-6478ea156cda None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:40.379907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 139/261] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/users/993f2ed94d2f4737b3477ca76762e011 => generated 283 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:40.385358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22713e-cbb3-4d85-9cfa-6478ea156cda None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.412149 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:40.465427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a4fdb-35be-47e5-9f64-285c846c75ac None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.529234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.529766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.530046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.530316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.531006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:40.531310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:40.532710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:40.548431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 123/262] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:40.554131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.576726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22713e-cbb3-4d85-9cfa-6478ea156cda None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.636898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073b9b-73ee-4678-8372-da5fbc364fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.637508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073b9b-73ee-4678-8372-da5fbc364fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.637789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073b9b-73ee-4678-8372-da5fbc364fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.638064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073b9b-73ee-4678-8372-da5fbc364fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.725950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c5abb-263d-4921-b9a8-7cea95ac8a14 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.756485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:40.775087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073b9b-73ee-4678-8372-da5fbc364fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:40.780052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073b9b-73ee-4678-8372-da5fbc364fd2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:40.798681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274be6bf-d5f8-43d3-ae5e-2a596c8c64f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.799344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274be6bf-d5f8-43d3-ae5e-2a596c8c64f5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.799624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274be6bf-d5f8-43d3-ae5e-2a596c8c64f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.799895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274be6bf-d5f8-43d3-ae5e-2a596c8c64f5 None admin] PATH_INFO: `/v3/users/993f2ed94d2f4737b3477ca76762e011` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.801037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274be6bf-d5f8-43d3-ae5e-2a596c8c64f5 None admin] RBAC: Authorizing `identity:get_user(user_id=993f2ed94d2f4737b3477ca76762e011)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:40.802498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274be6bf-d5f8-43d3-ae5e-2a596c8c64f5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:40.804427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 124/263] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/users/993f2ed94d2f4737b3477ca76762e011 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:40.812545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274be6bf-d5f8-43d3-ae5e-2a596c8c64f5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.818312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:40.966754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 140/264] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:45:40.972755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073b9b-73ee-4678-8372-da5fbc364fd2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:41.002989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274be6bf-d5f8-43d3-ae5e-2a596c8c64f5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.093378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5843-4e27-4de4-ae0d-aca8a9aabefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.094046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5843-4e27-4de4-ae0d-aca8a9aabefb None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.094333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5843-4e27-4de4-ae0d-aca8a9aabefb None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.094608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d5843-4e27-4de4-ae0d-aca8a9aabefb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.210227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.223084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073b9b-73ee-4678-8372-da5fbc364fd2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.235337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d5843-4e27-4de4-ae0d-aca8a9aabefb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.236815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d5843-4e27-4de4-ae0d-aca8a9aabefb None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.239521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.295620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627f2fe-2cd0-4812-b395-f630476709bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.296275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627f2fe-2cd0-4812-b395-f630476709bd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.296556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627f2fe-2cd0-4812-b395-f630476709bd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.296829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627f2fe-2cd0-4812-b395-f630476709bd None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.298018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627f2fe-2cd0-4812-b395-f630476709bd None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.299596 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627f2fe-2cd0-4812-b395-f630476709bd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.301653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 141/265] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:41.308121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627f2fe-2cd0-4812-b395-f630476709bd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:41.432527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 125/266] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:41.438990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5843-4e27-4de4-ae0d-aca8a9aabefb None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:41.515935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627f2fe-2cd0-4812-b395-f630476709bd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.600281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc99f3-8a7b-4a3e-a0f4-402dca25a7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.601313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc99f3-8a7b-4a3e-a0f4-402dca25a7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.601825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc99f3-8a7b-4a3e-a0f4-402dca25a7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.602315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc99f3-8a7b-4a3e-a0f4-402dca25a7a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.603296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc99f3-8a7b-4a3e-a0f4-402dca25a7a1 None admin] RBAC: Adding query filter params (role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591, user.id=993f2ed94d2f4737b3477ca76762e011) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:41.603832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc99f3-8a7b-4a3e-a0f4-402dca25a7a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.605543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc99f3-8a7b-4a3e-a0f4-402dca25a7a1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.612573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 142/267] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:41 2022] GET /identity/v3/role_assignments?user.id=993f2ed94d2f4737b3477ca76762e011&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=2e40648646094078acac22f8eb94dd98 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:41.633474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d5843-4e27-4de4-ae0d-aca8a9aabefb None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.635882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.674751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4690-7054-4c9e-94c9-629d4014373c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.675389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4690-7054-4c9e-94c9-629d4014373c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.675890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4690-7054-4c9e-94c9-629d4014373c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.677528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 143/268] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:41.684718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c64a0-0e70-4e8b-97e9-4dd94b82e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.685364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c64a0-0e70-4e8b-97e9-4dd94b82e06b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.685652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c64a0-0e70-4e8b-97e9-4dd94b82e06b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.685927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c64a0-0e70-4e8b-97e9-4dd94b82e06b None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.687092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c64a0-0e70-4e8b-97e9-4dd94b82e06b None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.688538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c64a0-0e70-4e8b-97e9-4dd94b82e06b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.690654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 126/269] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:41 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:41.696951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be4690-7054-4c9e-94c9-629d4014373c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:41.704766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2262f3f-b74d-49cb-bfaf-7d9dcf0add51 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.705052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2262f3f-b74d-49cb-bfaf-7d9dcf0add51 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.705323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2262f3f-b74d-49cb-bfaf-7d9dcf0add51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.732090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d2262f3f-b74d-49cb-bfaf-7d9dcf0add51 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:41.733591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d2262f3f-b74d-49cb-bfaf-7d9dcf0add51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:41.751305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.876807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 127/270] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:41.880947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3698d8-e96d-4fee-b069-9f2c37ada918 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.881451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3698d8-e96d-4fee-b069-9f2c37ada918 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.881868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3698d8-e96d-4fee-b069-9f2c37ada918 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.884002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 128/271] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:41.889755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f52e12-85b9-419d-948b-a3da31f18c46 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.890259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f52e12-85b9-419d-948b-a3da31f18c46 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.890750 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f52e12-85b9-419d-948b-a3da31f18c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.896136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be4690-7054-4c9e-94c9-629d4014373c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.913319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a0f52e12-85b9-419d-948b-a3da31f18c46 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:41.915696 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a0f52e12-85b9-419d-948b-a3da31f18c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:41.955040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02f779-d0f3-4206-928d-e22f5f677d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.955580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02f779-d0f3-4206-928d-e22f5f677d99 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.955866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02f779-d0f3-4206-928d-e22f5f677d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.956141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02f779-d0f3-4206-928d-e22f5f677d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.958913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02f779-d0f3-4206-928d-e22f5f677d99 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591, user.id=993f2ed94d2f4737b3477ca76762e011) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:41.958913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02f779-d0f3-4206-928d-e22f5f677d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.959174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02f779-d0f3-4206-928d-e22f5f677d99 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.967927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 144/272] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:41 2022] GET /identity/v3/role_assignments?user.id=993f2ed94d2f4737b3477ca76762e011&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:42.034447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 129/273] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:42.059961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af095e2-5045-4f23-b0d2-361735d57a8f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.060267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af095e2-5045-4f23-b0d2-361735d57a8f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.060526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af095e2-5045-4f23-b0d2-361735d57a8f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.062641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 145/274] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:42.068537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc375c6-20d2-4eac-af57-febb7c7b89b4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.068868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc375c6-20d2-4eac-af57-febb7c7b89b4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.069090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc375c6-20d2-4eac-af57-febb7c7b89b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.096422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-0cc375c6-20d2-4eac-af57-febb7c7b89b4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:42.097621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0cc375c6-20d2-4eac-af57-febb7c7b89b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:42.130855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.184876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 130/275] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:42.302125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af095e2-5045-4f23-b0d2-361735d57a8f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.345030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc375c6-20d2-4eac-af57-febb7c7b89b4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.432335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.449866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af095e2-5045-4f23-b0d2-361735d57a8f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.472813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc375c6-20d2-4eac-af57-febb7c7b89b4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.485686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584df3cb-208f-4e26-bcb5-19ebc7db14e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.486624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584df3cb-208f-4e26-bcb5-19ebc7db14e6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.486931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584df3cb-208f-4e26-bcb5-19ebc7db14e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.487209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584df3cb-208f-4e26-bcb5-19ebc7db14e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.503336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584df3cb-208f-4e26-bcb5-19ebc7db14e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-584df3cb-208f-4e26-bcb5-19ebc7db14e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:42.508406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:42.509368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:42.510244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 146/276] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:42.522090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584df3cb-208f-4e26-bcb5-19ebc7db14e6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.527955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1619bbd-3012-44b4-a0d1-bbfec767c1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.528907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619bbd-3012-44b4-a0d1-bbfec767c1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.529390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619bbd-3012-44b4-a0d1-bbfec767c1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.529859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619bbd-3012-44b4-a0d1-bbfec767c1f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.531237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1619bbd-3012-44b4-a0d1-bbfec767c1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:42.533185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1619bbd-3012-44b4-a0d1-bbfec767c1f7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:42.535266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 131/277] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:42.536234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1619bbd-3012-44b4-a0d1-bbfec767c1f7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.571074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.629634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584df3cb-208f-4e26-bcb5-19ebc7db14e6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.642738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1619bbd-3012-44b4-a0d1-bbfec767c1f7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.662729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.663359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.663684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.664475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.665126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:42.665421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:42.666785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:42.674496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 147/278] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:42.675282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.675895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39df1d3-1bdc-434e-931d-96ecf09fed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.676701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39df1d3-1bdc-434e-931d-96ecf09fed64 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.677122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39df1d3-1bdc-434e-931d-96ecf09fed64 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.677532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39df1d3-1bdc-434e-931d-96ecf09fed64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.687691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39df1d3-1bdc-434e-931d-96ecf09fed64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:42.691714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b39df1d3-1bdc-434e-931d-96ecf09fed64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:42.693536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:42.694419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:42.695267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 132/279] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:42.696177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaea94-d3ef-413b-8a33-6c81af254d6e None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.696267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaea94-d3ef-413b-8a33-6c81af254d6e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.696510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaea94-d3ef-413b-8a33-6c81af254d6e None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.697829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 133/280] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:42.698375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaea94-d3ef-413b-8a33-6c81af254d6e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.777508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98c635-0373-416a-a12e-528b69da5151 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.789913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.799893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaea94-d3ef-413b-8a33-6c81af254d6e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.810909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab64da-67c0-464a-995d-74632736b461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.811531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab64da-67c0-464a-995d-74632736b461 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.811812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab64da-67c0-464a-995d-74632736b461 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.812081 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab64da-67c0-464a-995d-74632736b461 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.812722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab64da-67c0-464a-995d-74632736b461 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:42.814406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab64da-67c0-464a-995d-74632736b461 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:42.833972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 148/281] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:42 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:42.835214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6411c-4229-4c03-b2d8-03134ac3f2db None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.836021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6411c-4229-4c03-b2d8-03134ac3f2db None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.836295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6411c-4229-4c03-b2d8-03134ac3f2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.836426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4bacd-5040-4198-9c70-ef21d6d7c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.837019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4bacd-5040-4198-9c70-ef21d6d7c82d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.837305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4bacd-5040-4198-9c70-ef21d6d7c82d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.837579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4bacd-5040-4198-9c70-ef21d6d7c82d None admin] PATH_INFO: `/v3/users/1f9fc9d55d1644fbb1f7cb663969f91d` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.855232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-eee6411c-4229-4c03-b2d8-03134ac3f2db None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:42.855587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4bacd-5040-4198-9c70-ef21d6d7c82d None admin] RBAC: Authorizing `identity:get_user(user_id=1f9fc9d55d1644fbb1f7cb663969f91d)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:42.856451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eee6411c-4229-4c03-b2d8-03134ac3f2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:42.857064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4bacd-5040-4198-9c70-ef21d6d7c82d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:42.858883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 134/282] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/users/1f9fc9d55d1644fbb1f7cb663969f91d => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:42.863978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4bacd-5040-4198-9c70-ef21d6d7c82d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.935949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.942317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 149/283] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:42.943355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6411c-4229-4c03-b2d8-03134ac3f2db None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.001992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4bacd-5040-4198-9c70-ef21d6d7c82d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.045533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.047133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.047599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.048053 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.048789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:43.049416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.050821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.060936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 135/284] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.066584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.069710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6411c-4229-4c03-b2d8-03134ac3f2db None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.083152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.116732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.117797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.118078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.118350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.119125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] RBAC: Adding query filter params (role.id=aa4ffe964b624a329cecff46b0a2c131, user.id=1f9fc9d55d1644fbb1f7cb663969f91d, scope.system=all) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:43.119434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.120901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.130287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 150/285] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/role_assignments?user.id=1f9fc9d55d1644fbb1f7cb663969f91d&scope.system=all&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.200700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6a55-e825-4de1-aa42-def5114f4599 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.226835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.244086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b866a40-dc75-4437-a051-ed8597bdb491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.244758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b866a40-dc75-4437-a051-ed8597bdb491 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.245086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b866a40-dc75-4437-a051-ed8597bdb491 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.245418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b866a40-dc75-4437-a051-ed8597bdb491 None admin] PATH_INFO: `/v3/users/0edd1a05d8314510975c83747cf0b430` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.247159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b866a40-dc75-4437-a051-ed8597bdb491 None admin] RBAC: Authorizing `identity:get_user(user_id=0edd1a05d8314510975c83747cf0b430)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.251306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b866a40-dc75-4437-a051-ed8597bdb491 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.251306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 136/286] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/users/0edd1a05d8314510975c83747cf0b430 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.255791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b866a40-dc75-4437-a051-ed8597bdb491 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.308785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.355517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.356993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd5d2-9ffd-4861-9dc5-0b1b5a23d3bd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.379114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b866a40-dc75-4437-a051-ed8597bdb491 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.394680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195889e-7031-441a-a7b7-c4d5a785f6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.395526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195889e-7031-441a-a7b7-c4d5a785f6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.395911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195889e-7031-441a-a7b7-c4d5a785f6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.396284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195889e-7031-441a-a7b7-c4d5a785f6d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.414917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195889e-7031-441a-a7b7-c4d5a785f6d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:43.415059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec01a8-9209-4844-bb7c-5c767f842c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.415059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec01a8-9209-4844-bb7c-5c767f842c8e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.415059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec01a8-9209-4844-bb7c-5c767f842c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.415059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec01a8-9209-4844-bb7c-5c767f842c8e None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.416534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec01a8-9209-4844-bb7c-5c767f842c8e None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.418028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec01a8-9209-4844-bb7c-5c767f842c8e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3195889e-7031-441a-a7b7-c4d5a785f6d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:43.419267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:43.420307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:43.421365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:43.423168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 137/287] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.423239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 151/288] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:43.426421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec01a8-9209-4844-bb7c-5c767f842c8e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.434919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195889e-7031-441a-a7b7-c4d5a785f6d2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.531533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec01a8-9209-4844-bb7c-5c767f842c8e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.537919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195889e-7031-441a-a7b7-c4d5a785f6d2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.564609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.567000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076b0f7-f070-4b02-afee-34b75d02793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.567764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076b0f7-f070-4b02-afee-34b75d02793f None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.568142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076b0f7-f070-4b02-afee-34b75d02793f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.568608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076b0f7-f070-4b02-afee-34b75d02793f None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/users/0edd1a05d8314510975c83747cf0b430/roles/8aad1777fec94e0b8c3dc239ae4864c6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.572075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076b0f7-f070-4b02-afee-34b75d02793f None admin] RBAC: Authorizing `identity:create_grant(project_id=fcc729b56f9c4ffb9e49f733ba16d591, user_id=0edd1a05d8314510975c83747cf0b430, role_id=8aad1777fec94e0b8c3dc239ae4864c6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.573647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076b0f7-f070-4b02-afee-34b75d02793f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.584748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.585984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.586259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.586814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.594828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:43.595169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.595623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 138/289] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:43 2022] PUT /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/users/0edd1a05d8314510975c83747cf0b430/roles/8aad1777fec94e0b8c3dc239ae4864c6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:43.596671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.613356 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 152/290] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:43.619542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076b0f7-f070-4b02-afee-34b75d02793f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.630042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.766184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076b0f7-f070-4b02-afee-34b75d02793f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.821745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9cd4e-00d3-4021-9691-c1ccbfd55458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.822803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cd4e-00d3-4021-9691-c1ccbfd55458 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.823342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cd4e-00d3-4021-9691-c1ccbfd55458 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.823854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cd4e-00d3-4021-9691-c1ccbfd55458 None admin] PATH_INFO: `/v3/users/69a894572e7742348231959c24b7897c` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.827643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9cd4e-00d3-4021-9691-c1ccbfd55458 None admin] RBAC: Authorizing `identity:get_user(user_id=69a894572e7742348231959c24b7897c)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.829401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9cd4e-00d3-4021-9691-c1ccbfd55458 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.831547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 139/291] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/users/69a894572e7742348231959c24b7897c => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.837279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.950608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3713b8-e580-4680-aaae-7cd01203c484 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.994546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac366e0-07f0-4bb4-aa7e-dc5aab3f385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.995198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac366e0-07f0-4bb4-aa7e-dc5aab3f385a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.995482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac366e0-07f0-4bb4-aa7e-dc5aab3f385a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.995755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac366e0-07f0-4bb4-aa7e-dc5aab3f385a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.072470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:44.086435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac366e0-07f0-4bb4-aa7e-dc5aab3f385a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.087877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac366e0-07f0-4bb4-aa7e-dc5aab3f385a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.235818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 153/292] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:44.241416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9cd4e-00d3-4021-9691-c1ccbfd55458 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:44.342527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:44.377577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9cd4e-00d3-4021-9691-c1ccbfd55458 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.420034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40380b-529b-43c9-94fd-37ec0f466e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.420673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40380b-529b-43c9-94fd-37ec0f466e04 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.420951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40380b-529b-43c9-94fd-37ec0f466e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.421222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40380b-529b-43c9-94fd-37ec0f466e04 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.422375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40380b-529b-43c9-94fd-37ec0f466e04 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.423840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40380b-529b-43c9-94fd-37ec0f466e04 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.425441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 140/293] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:44 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:44.431961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac366e0-07f0-4bb4-aa7e-dc5aab3f385a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:44.562837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:44.582960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac366e0-07f0-4bb4-aa7e-dc5aab3f385a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.635914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbba0e-6a68-4b65-9a78-2d5eb6822c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.636912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbba0e-6a68-4b65-9a78-2d5eb6822c86 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.637383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbba0e-6a68-4b65-9a78-2d5eb6822c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.637879 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbba0e-6a68-4b65-9a78-2d5eb6822c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.638834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbba0e-6a68-4b65-9a78-2d5eb6822c86 None admin] RBAC: Adding query filter params (role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591, user.id=69a894572e7742348231959c24b7897c) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:44.639363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbba0e-6a68-4b65-9a78-2d5eb6822c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.640963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbba0e-6a68-4b65-9a78-2d5eb6822c86 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.652400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 154/294] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:44 2022] GET /identity/v3/role_assignments?user.id=69a894572e7742348231959c24b7897c&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:45.649218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aced8e-1f6a-41d7-a599-7ea2e0465a56 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.649719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aced8e-1f6a-41d7-a599-7ea2e0465a56 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.649825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aced8e-1f6a-41d7-a599-7ea2e0465a56 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.651271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 141/295] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:45.652134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e8a92-ff12-48b1-9e54-f512922b3bba None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.652507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e8a92-ff12-48b1-9e54-f512922b3bba None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.652722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e8a92-ff12-48b1-9e54-f512922b3bba None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.654317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 155/296] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:45.655912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fb19e-d4db-4232-9a34-9d01287602f0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.656197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fb19e-d4db-4232-9a34-9d01287602f0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.656469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fb19e-d4db-4232-9a34-9d01287602f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.658984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234f645-dc7c-4a4d-8cea-8615c4af1237 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.659265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234f645-dc7c-4a4d-8cea-8615c4af1237 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.659654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234f645-dc7c-4a4d-8cea-8615c4af1237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.676860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f56fb19e-d4db-4232-9a34-9d01287602f0 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:45.677478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3234f645-dc7c-4a4d-8cea-8615c4af1237 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:45.678263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f56fb19e-d4db-4232-9a34-9d01287602f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:45.678616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3234f645-dc7c-4a4d-8cea-8615c4af1237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:45.750448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 142/297] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:45.751964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc3fa2-bd36-419d-aaa1-9668b006d57a None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.752257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc3fa2-bd36-419d-aaa1-9668b006d57a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.752533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc3fa2-bd36-419d-aaa1-9668b006d57a None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.753775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 143/298] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:45.759801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe05e0-832c-4d65-bb31-19d1f96862e3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.760090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe05e0-832c-4d65-bb31-19d1f96862e3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.760364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe05e0-832c-4d65-bb31-19d1f96862e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.764358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 156/299] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:45.780821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-0bfe05e0-832c-4d65-bb31-19d1f96862e3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:45.782147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0bfe05e0-832c-4d65-bb31-19d1f96862e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:45.859027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 144/300] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:46.026579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3234f645-dc7c-4a4d-8cea-8615c4af1237 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.038939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe05e0-832c-4d65-bb31-19d1f96862e3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.042425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.120219 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.142519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3234f645-dc7c-4a4d-8cea-8615c4af1237 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.161684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe05e0-832c-4d65-bb31-19d1f96862e3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.179746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d8a14-db0a-4f65-8ab8-353f4d446959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.180390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d8a14-db0a-4f65-8ab8-353f4d446959 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.180668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d8a14-db0a-4f65-8ab8-353f4d446959 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.180941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d8a14-db0a-4f65-8ab8-353f4d446959 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.182040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d8a14-db0a-4f65-8ab8-353f4d446959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.183508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d8a14-db0a-4f65-8ab8-353f4d446959 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.185073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 157/301] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:46.190470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d8a14-db0a-4f65-8ab8-353f4d446959 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.198910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaba97b-2c90-4b83-bcf3-ed0af7a22406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.199562 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaba97b-2c90-4b83-bcf3-ed0af7a22406 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.199845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaba97b-2c90-4b83-bcf3-ed0af7a22406 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.200121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaba97b-2c90-4b83-bcf3-ed0af7a22406 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.211364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaba97b-2c90-4b83-bcf3-ed0af7a22406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-7aaba97b-2c90-4b83-bcf3-ed0af7a22406 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:46.216272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:46.217125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 145/302] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:46.222829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaba97b-2c90-4b83-bcf3-ed0af7a22406 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.286301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.310055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d8a14-db0a-4f65-8ab8-353f4d446959 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.326619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaba97b-2c90-4b83-bcf3-ed0af7a22406 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.345211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83b8c-bfef-4b19-90e4-dca47eb1491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.345926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83b8c-bfef-4b19-90e4-dca47eb1491f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.346200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83b8c-bfef-4b19-90e4-dca47eb1491f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.346468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83b8c-bfef-4b19-90e4-dca47eb1491f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.356996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c83b8c-bfef-4b19-90e4-dca47eb1491f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:46.361103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa448b-a741-4f64-9e17-f9049f45502a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.361707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa448b-a741-4f64-9e17-f9049f45502a None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-88c83b8c-bfef-4b19-90e4-dca47eb1491f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:46.361833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:46.362817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:46.363637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:46.364305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa448b-a741-4f64-9e17-f9049f45502a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.364305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa448b-a741-4f64-9e17-f9049f45502a None admin] PATH_INFO: `/v3/groups` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.364305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa448b-a741-4f64-9e17-f9049f45502a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.364467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 158/303] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:46.364467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83b8c-bfef-4b19-90e4-dca47eb1491f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.364588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa448b-a741-4f64-9e17-f9049f45502a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.377975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.380879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 146/304] 10.222.0.52 () {60 vars in 1134 bytes} [Mon Oct 10 11:45:46 2022] POST /identity/v3/groups => generated 229 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:46.382116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916906-707b-4828-8ba1-2a9daa3933da None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.383127 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916906-707b-4828-8ba1-2a9daa3933da None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.383320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916906-707b-4828-8ba1-2a9daa3933da None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.384510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 147/305] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:46.385203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5916906-707b-4828-8ba1-2a9daa3933da None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.493195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83b8c-bfef-4b19-90e4-dca47eb1491f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.505518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5916906-707b-4828-8ba1-2a9daa3933da None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.530322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.533607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e63fc2-1518-4ef4-90f5-0b610b46c43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.534314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e63fc2-1518-4ef4-90f5-0b610b46c43d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.534505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e63fc2-1518-4ef4-90f5-0b610b46c43d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.534775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e63fc2-1518-4ef4-90f5-0b610b46c43d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.535478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e63fc2-1518-4ef4-90f5-0b610b46c43d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:46.536008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e63fc2-1518-4ef4-90f5-0b610b46c43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.537256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e63fc2-1518-4ef4-90f5-0b610b46c43d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.546020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.546841 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.546971 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 159/306] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:46.547994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f8e0b-b840-4aa2-b29d-7b9be3c397d0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.548283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f8e0b-b840-4aa2-b29d-7b9be3c397d0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.548556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f8e0b-b840-4aa2-b29d-7b9be3c397d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.551773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.552166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.552878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:46.553191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.555158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.567255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 148/307] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:46.567255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.567837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-227f8e0b-b840-4aa2-b29d-7b9be3c397d0 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:46.569028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-227f8e0b-b840-4aa2-b29d-7b9be3c397d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:46.641483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.652825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 160/308] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:46.652825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f8e0b-b840-4aa2-b29d-7b9be3c397d0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.679141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c22d4-9c7e-4f93-ace4-46a914c03658 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.718365 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164979d-035f-45cd-a2a5-a4f35ef0b0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.719488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164979d-035f-45cd-a2a5-a4f35ef0b0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.720008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164979d-035f-45cd-a2a5-a4f35ef0b0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.720293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164979d-035f-45cd-a2a5-a4f35ef0b0f5 None admin] PATH_INFO: `/v3/users/9fb20fb6dd774623a52a3263e7f63309` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.737801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164979d-035f-45cd-a2a5-a4f35ef0b0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb20fb6dd774623a52a3263e7f63309)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.739774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164979d-035f-45cd-a2a5-a4f35ef0b0f5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.743054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 149/309] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/users/9fb20fb6dd774623a52a3263e7f63309 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:46.748447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164979d-035f-45cd-a2a5-a4f35ef0b0f5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.764365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f8e0b-b840-4aa2-b29d-7b9be3c397d0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.800123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a291c-14b0-4ae0-b669-3ee3d5cb606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.800685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a291c-14b0-4ae0-b669-3ee3d5cb606b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.800960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a291c-14b0-4ae0-b669-3ee3d5cb606b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.801224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a291c-14b0-4ae0-b669-3ee3d5cb606b None admin] PATH_INFO: `/v3/users/1f9fc9d55d1644fbb1f7cb663969f91d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.802334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a291c-14b0-4ae0-b669-3ee3d5cb606b None admin] RBAC: Authorizing `identity:get_user(user_id=1f9fc9d55d1644fbb1f7cb663969f91d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.802551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.803867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a291c-14b0-4ae0-b669-3ee3d5cb606b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.806016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 161/310] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/users/1f9fc9d55d1644fbb1f7cb663969f91d => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:46.812186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a291c-14b0-4ae0-b669-3ee3d5cb606b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.872523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164979d-035f-45cd-a2a5-a4f35ef0b0f5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.892340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.908722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.909326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.909604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.909872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.910647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] RBAC: Adding query filter params (role.id=8aad1777fec94e0b8c3dc239ae4864c6, user.id=9fb20fb6dd774623a52a3263e7f63309, scope.system=all) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:46.910995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.912482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.918057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 150/311] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/role_assignments?user.id=9fb20fb6dd774623a52a3263e7f63309&scope.system=all&role.id=8aad1777fec94e0b8c3dc239ae4864c6 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:46.932800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a291c-14b0-4ae0-b669-3ee3d5cb606b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.947206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.980626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa7b8c-fdba-4f3e-ab74-934ecdfb845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.981704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa7b8c-fdba-4f3e-ab74-934ecdfb845a None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.981995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa7b8c-fdba-4f3e-ab74-934ecdfb845a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.982281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa7b8c-fdba-4f3e-ab74-934ecdfb845a None admin] PATH_INFO: `/v3/system/users/1f9fc9d55d1644fbb1f7cb663969f91d/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.984260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa7b8c-fdba-4f3e-ab74-934ecdfb845a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f9fc9d55d1644fbb1f7cb663969f91d, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.985865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa7b8c-fdba-4f3e-ab74-934ecdfb845a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:47.005463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 162/312] 10.222.0.52 () {58 vars in 1334 bytes} [Mon Oct 10 11:45:46 2022] PUT /identity/v3/system/users/1f9fc9d55d1644fbb1f7cb663969f91d/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:47.078206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a2fa9-b09d-4236-a10b-1763393add0a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:47.087487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:47.121247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377b840-c08e-4336-bb53-b8be53ab383a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:47.122240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b840-c08e-4336-bb53-b8be53ab383a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.122711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b840-c08e-4336-bb53-b8be53ab383a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.123199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377b840-c08e-4336-bb53-b8be53ab383a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.135959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377b840-c08e-4336-bb53-b8be53ab383a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b377b840-c08e-4336-bb53-b8be53ab383a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:47.141306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:47.142248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:47.143038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:47.145945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 151/313] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:47.152879 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa7b8c-fdba-4f3e-ab74-934ecdfb845a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:47.267196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa7b8c-fdba-4f3e-ab74-934ecdfb845a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:47.298312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:47.304195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:47.304825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.305488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.306124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.306793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:47.307130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:47.309046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:47.319739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 163/314] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:47.325779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377b840-c08e-4336-bb53-b8be53ab383a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:47.441433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377b840-c08e-4336-bb53-b8be53ab383a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:47.478246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84fd78-c917-4878-bde0-be09d05621e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:47.478963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84fd78-c917-4878-bde0-be09d05621e2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.479284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84fd78-c917-4878-bde0-be09d05621e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.479560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84fd78-c917-4878-bde0-be09d05621e2 None admin] PATH_INFO: `/v3/users/0edd1a05d8314510975c83747cf0b430` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.480700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84fd78-c917-4878-bde0-be09d05621e2 None admin] RBAC: Authorizing `identity:get_user(user_id=0edd1a05d8314510975c83747cf0b430)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:47.482132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84fd78-c917-4878-bde0-be09d05621e2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:47.483738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 152/315] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:47 2022] GET /identity/v3/users/0edd1a05d8314510975c83747cf0b430 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:47.489602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:47.600229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3165802-78d8-4295-b5b7-1fd4a6d3d98d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:47.635920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301c974-941c-44f2-bbe3-8b351bcec71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:47.636811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c974-941c-44f2-bbe3-8b351bcec71f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.637519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c974-941c-44f2-bbe3-8b351bcec71f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.637985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301c974-941c-44f2-bbe3-8b351bcec71f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.707390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:47.720002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301c974-941c-44f2-bbe3-8b351bcec71f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:47.721710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301c974-941c-44f2-bbe3-8b351bcec71f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:47.823766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 164/316] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:47.829276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84fd78-c917-4878-bde0-be09d05621e2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:47.875399 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:47.943662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84fd78-c917-4878-bde0-be09d05621e2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:47.978269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bff69-e854-4d02-b375-7dd91f76ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:47.978874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bff69-e854-4d02-b375-7dd91f76ed2f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.979176 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bff69-e854-4d02-b375-7dd91f76ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.979498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bff69-e854-4d02-b375-7dd91f76ed2f None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.980609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bff69-e854-4d02-b375-7dd91f76ed2f None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:47.982103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bff69-e854-4d02-b375-7dd91f76ed2f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:47.984092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 153/317] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:47 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:47.989757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301c974-941c-44f2-bbe3-8b351bcec71f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:48.087298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:48.094281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301c974-941c-44f2-bbe3-8b351bcec71f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:48.133259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b6818-1cc0-43f9-b6c8-333efa37a6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:48.133860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b6818-1cc0-43f9-b6c8-333efa37a6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.134141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b6818-1cc0-43f9-b6c8-333efa37a6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.134413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b6818-1cc0-43f9-b6c8-333efa37a6c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.135164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b6818-1cc0-43f9-b6c8-333efa37a6c3 None admin] RBAC: Adding query filter params (role.id=8aad1777fec94e0b8c3dc239ae4864c6, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591, user.id=0edd1a05d8314510975c83747cf0b430) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:48.135469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b6818-1cc0-43f9-b6c8-333efa37a6c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:48.136887 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b6818-1cc0-43f9-b6c8-333efa37a6c3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:48.143003 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 165/318] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:47 2022] GET /identity/v3/role_assignments?user.id=0edd1a05d8314510975c83747cf0b430&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=8aad1777fec94e0b8c3dc239ae4864c6 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:48.710269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e9f5-969c-49e6-947b-5fbc67bc7409 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.710483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e9f5-969c-49e6-947b-5fbc67bc7409 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.710699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e9f5-969c-49e6-947b-5fbc67bc7409 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.712164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 154/319] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:48.717358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7c6e-4044-4328-b18d-11831e9641d2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.717808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7c6e-4044-4328-b18d-11831e9641d2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.717908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7c6e-4044-4328-b18d-11831e9641d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.736122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3caf7c6e-4044-4328-b18d-11831e9641d2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:48.737313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3caf7c6e-4044-4328-b18d-11831e9641d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:48.805851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 166/320] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:49.076472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e9f5-969c-49e6-947b-5fbc67bc7409 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.179356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e9f5-969c-49e6-947b-5fbc67bc7409 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.182530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2b1b-945a-4480-a1d5-ae589fa88377 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.182810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2b1b-945a-4480-a1d5-ae589fa88377 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.183181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:49.183440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2b1b-945a-4480-a1d5-ae589fa88377 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.185059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 167/321] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:49.193262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3db12d-0614-4f2c-97f2-3e2c6d2087c4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.193548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3db12d-0614-4f2c-97f2-3e2c6d2087c4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.194445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3db12d-0614-4f2c-97f2-3e2c6d2087c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.214548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0f3db12d-0614-4f2c-97f2-3e2c6d2087c4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:49.215772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f3db12d-0614-4f2c-97f2-3e2c6d2087c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:49.227510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d127b6-0e44-43e4-a6ec-c95835e63c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.228141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d127b6-0e44-43e4-a6ec-c95835e63c57 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.228421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d127b6-0e44-43e4-a6ec-c95835e63c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.228695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d127b6-0e44-43e4-a6ec-c95835e63c57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.229791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d127b6-0e44-43e4-a6ec-c95835e63c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:49.231257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d127b6-0e44-43e4-a6ec-c95835e63c57 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:49.232965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 155/322] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:49.238175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d127b6-0e44-43e4-a6ec-c95835e63c57 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.292811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 168/323] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:49.304803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58a818-16c8-4c00-989e-438704acf282 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.305081 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58a818-16c8-4c00-989e-438704acf282 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.305366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58a818-16c8-4c00-989e-438704acf282 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.306645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 169/324] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:49.311479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1519b-1fe2-44a6-b58b-288a93091235 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.311872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1519b-1fe2-44a6-b58b-288a93091235 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.312113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1519b-1fe2-44a6-b58b-288a93091235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.331678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-92c1519b-1fe2-44a6-b58b-288a93091235 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:49.332768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-92c1519b-1fe2-44a6-b58b-288a93091235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:49.346725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d127b6-0e44-43e4-a6ec-c95835e63c57 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.347198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:49.384854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720853b2-4553-4041-9792-0d1eaf867653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.385877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720853b2-4553-4041-9792-0d1eaf867653 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.386332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720853b2-4553-4041-9792-0d1eaf867653 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.386773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720853b2-4553-4041-9792-0d1eaf867653 None admin] PATH_INFO: `/v3/groups` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.387663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720853b2-4553-4041-9792-0d1eaf867653 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:49.389396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720853b2-4553-4041-9792-0d1eaf867653 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:49.402317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 170/325] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:49.409910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 156/326] 10.222.0.52 () {60 vars in 1134 bytes} [Mon Oct 10 11:45:49 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:45:49.569347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1519b-1fe2-44a6-b58b-288a93091235 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.673002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1519b-1fe2-44a6-b58b-288a93091235 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.673165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720853b2-4553-4041-9792-0d1eaf867653 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.718690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc4204-4c0d-4c75-85ae-5977401206c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.719238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:49.719468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc4204-4c0d-4c75-85ae-5977401206c6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.719671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc4204-4c0d-4c75-85ae-5977401206c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.719910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc4204-4c0d-4c75-85ae-5977401206c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.731390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc4204-4c0d-4c75-85ae-5977401206c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-75dc4204-4c0d-4c75-85ae-5977401206c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:49.736233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:49.737115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:49.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 171/327] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:49.743368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc4204-4c0d-4c75-85ae-5977401206c6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.758842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:49.819580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720853b2-4553-4041-9792-0d1eaf867653 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.863843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80842cbe-d0be-45b7-9ac8-891be1f90deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.864487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80842cbe-d0be-45b7-9ac8-891be1f90deb None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.864767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80842cbe-d0be-45b7-9ac8-891be1f90deb None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.865035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80842cbe-d0be-45b7-9ac8-891be1f90deb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.879162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80842cbe-d0be-45b7-9ac8-891be1f90deb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:49.882632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc4204-4c0d-4c75-85ae-5977401206c6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-80842cbe-d0be-45b7-9ac8-891be1f90deb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:49.886115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:49.887014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:49.887803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:49.889670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 157/328] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:49.896854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80842cbe-d0be-45b7-9ac8-891be1f90deb None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.919226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.919834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.920116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.920383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.921049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:49.921346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:49.923285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:49.933592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 172/329] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:49.939360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.006192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:50.011521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:50.018764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80842cbe-d0be-45b7-9ac8-891be1f90deb None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.044652 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35923f8e-4b80-427c-a4bc-74d2e480cb58 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.056601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.057225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.057512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.057793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.058471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:50.058778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.060227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.072251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 158/330] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:50.077362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.082800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549635e-f93b-4c57-adf6-213464c3fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.083486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549635e-f93b-4c57-adf6-213464c3fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.083766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549635e-f93b-4c57-adf6-213464c3fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.084039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549635e-f93b-4c57-adf6-213464c3fcb4 None admin] PATH_INFO: `/v3/users/9fb20fb6dd774623a52a3263e7f63309` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.085703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549635e-f93b-4c57-adf6-213464c3fcb4 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb20fb6dd774623a52a3263e7f63309)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.087194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549635e-f93b-4c57-adf6-213464c3fcb4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.093441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 173/331] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/users/9fb20fb6dd774623a52a3263e7f63309 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:50.094266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549635e-f93b-4c57-adf6-213464c3fcb4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.191967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb5133-b7e9-4293-81cd-007ffd1e4c00 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.209208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549635e-f93b-4c57-adf6-213464c3fcb4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.230668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c550d-80da-473c-8e9b-d81837d06f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.231462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c550d-80da-473c-8e9b-d81837d06f01 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.231752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c550d-80da-473c-8e9b-d81837d06f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.232028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c550d-80da-473c-8e9b-d81837d06f01 None admin] PATH_INFO: `/v3/users/1f9fc9d55d1644fbb1f7cb663969f91d` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.233271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c550d-80da-473c-8e9b-d81837d06f01 None admin] RBAC: Authorizing `identity:get_user(user_id=1f9fc9d55d1644fbb1f7cb663969f91d)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.234847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c550d-80da-473c-8e9b-d81837d06f01 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.237040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 159/332] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:50 2022] GET /identity/v3/users/1f9fc9d55d1644fbb1f7cb663969f91d => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:50.242654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c550d-80da-473c-8e9b-d81837d06f01 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.246325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2178be6-b5a8-4148-9b8a-0e0cef0d1aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.247007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2178be6-b5a8-4148-9b8a-0e0cef0d1aca None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.247287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2178be6-b5a8-4148-9b8a-0e0cef0d1aca None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.247557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2178be6-b5a8-4148-9b8a-0e0cef0d1aca None admin] PATH_INFO: `/v3/system/users/9fb20fb6dd774623a52a3263e7f63309/roles/8aad1777fec94e0b8c3dc239ae4864c6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.248988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2178be6-b5a8-4148-9b8a-0e0cef0d1aca None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9fb20fb6dd774623a52a3263e7f63309, role_id=8aad1777fec94e0b8c3dc239ae4864c6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.250368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2178be6-b5a8-4148-9b8a-0e0cef0d1aca None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.260411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 174/333] 10.222.0.52 () {58 vars in 1334 bytes} [Mon Oct 10 11:45:50 2022] PUT /identity/v3/system/users/9fb20fb6dd774623a52a3263e7f63309/roles/8aad1777fec94e0b8c3dc239ae4864c6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:50.271287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:50.364952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c550d-80da-473c-8e9b-d81837d06f01 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.409773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452525fd-6af8-4bfa-b683-f6faa30e2e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.410400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452525fd-6af8-4bfa-b683-f6faa30e2e99 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.410680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452525fd-6af8-4bfa-b683-f6faa30e2e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.410992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452525fd-6af8-4bfa-b683-f6faa30e2e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.414580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452525fd-6af8-4bfa-b683-f6faa30e2e99 None admin] RBAC: Adding query filter params (role.id=aa4ffe964b624a329cecff46b0a2c131, user.id=1f9fc9d55d1644fbb1f7cb663969f91d, scope.system=all) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:50.414580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452525fd-6af8-4bfa-b683-f6faa30e2e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.414580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452525fd-6af8-4bfa-b683-f6faa30e2e99 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.428247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 160/334] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Oct 10 11:45:50 2022] GET /identity/v3/role_assignments?user.id=1f9fc9d55d1644fbb1f7cb663969f91d&scope.system=all&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:51.885664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d477-2cbf-44ad-88a9-295701a30353 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:51.886676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d477-2cbf-44ad-88a9-295701a30353 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:51.887181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d477-2cbf-44ad-88a9-295701a30353 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.889063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 175/335] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:51.894172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e822a3-85cd-40e3-b480-1cdb1ac95afa None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:51.895045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e822a3-85cd-40e3-b480-1cdb1ac95afa None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:51.895521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e822a3-85cd-40e3-b480-1cdb1ac95afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.918975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-06e822a3-85cd-40e3-b480-1cdb1ac95afa None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:51.920575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-06e822a3-85cd-40e3-b480-1cdb1ac95afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:52.004804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 161/336] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:52.344401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a42a10-406a-4117-aacf-d934eab9985c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.345086 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a42a10-406a-4117-aacf-d934eab9985c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.345590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a42a10-406a-4117-aacf-d934eab9985c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.347282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 176/337] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:52.360574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a487ce-11ce-479f-a737-b47985e5540a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.361161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a487ce-11ce-479f-a737-b47985e5540a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.361633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a487ce-11ce-479f-a737-b47985e5540a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.363704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a42a10-406a-4117-aacf-d934eab9985c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.367224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:52.396382 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-89a487ce-11ce-479f-a737-b47985e5540a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:52.398173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-89a487ce-11ce-479f-a737-b47985e5540a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:52.405036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:52.527160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 162/338] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:52.533493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a42a10-406a-4117-aacf-d934eab9985c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.571856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67b3cb-5610-40b2-82bc-627029dfe605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:52.572529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67b3cb-5610-40b2-82bc-627029dfe605 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.572811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67b3cb-5610-40b2-82bc-627029dfe605 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.573079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67b3cb-5610-40b2-82bc-627029dfe605 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.592217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67b3cb-5610-40b2-82bc-627029dfe605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ec67b3cb-5610-40b2-82bc-627029dfe605 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:52.597535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:52.598373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:52.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 177/339] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:52.605056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a487ce-11ce-479f-a737-b47985e5540a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.724422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a487ce-11ce-479f-a737-b47985e5540a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.764831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:52.765444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.765720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.765986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.766638 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:52.766963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:52.768312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:52.777248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:52.780857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 163/340] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:52.786752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67b3cb-5610-40b2-82bc-627029dfe605 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.837029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.858502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:52.908761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67b3cb-5610-40b2-82bc-627029dfe605 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.970312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c067f82-8cd0-4ad5-b717-01ad3ea1504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:52.971307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c067f82-8cd0-4ad5-b717-01ad3ea1504b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.971841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c067f82-8cd0-4ad5-b717-01ad3ea1504b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.972387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c067f82-8cd0-4ad5-b717-01ad3ea1504b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.982012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aae57-06fe-4c8a-9425-b74e8414c25b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.061551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d307e-be95-4662-bdbd-ebe319a93df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.062198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d307e-be95-4662-bdbd-ebe319a93df6 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.062471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d307e-be95-4662-bdbd-ebe319a93df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.062736 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d307e-be95-4662-bdbd-ebe319a93df6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.086343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d307e-be95-4662-bdbd-ebe319a93df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-fe9d307e-be95-4662-bdbd-ebe319a93df6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:53.092986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:53.093886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:53.094650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:53.097106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c067f82-8cd0-4ad5-b717-01ad3ea1504b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.098789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c067f82-8cd0-4ad5-b717-01ad3ea1504b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.107378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 164/341] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:53.119429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d307e-be95-4662-bdbd-ebe319a93df6 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.205461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.238393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.264071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 178/342] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:53.265819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f0fa1-7126-472c-9c0f-6acf82766c3e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.267066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f0fa1-7126-472c-9c0f-6acf82766c3e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.267553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f0fa1-7126-472c-9c0f-6acf82766c3e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.269217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 179/343] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:53.270192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f0fa1-7126-472c-9c0f-6acf82766c3e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.330686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d307e-be95-4662-bdbd-ebe319a93df6 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.404668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f0fa1-7126-472c-9c0f-6acf82766c3e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.407538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826826b5-718d-4204-a410-dd2dfbcf1732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.408161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826826b5-718d-4204-a410-dd2dfbcf1732 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.408430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826826b5-718d-4204-a410-dd2dfbcf1732 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.408694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826826b5-718d-4204-a410-dd2dfbcf1732 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.409356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826826b5-718d-4204-a410-dd2dfbcf1732 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:53.409651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826826b5-718d-4204-a410-dd2dfbcf1732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.411051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826826b5-718d-4204-a410-dd2dfbcf1732 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.430995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 165/344] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:53.432289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cad019-ce72-41dc-a5b9-cf1d415b490e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.434353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cad019-ce72-41dc-a5b9-cf1d415b490e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.434622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cad019-ce72-41dc-a5b9-cf1d415b490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.463544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f8cad019-ce72-41dc-a5b9-cf1d415b490e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:53.464880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f8cad019-ce72-41dc-a5b9-cf1d415b490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:53.471573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb638d-2ff5-4131-a958-81353f0bb282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.473080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb638d-2ff5-4131-a958-81353f0bb282 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.473578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb638d-2ff5-4131-a958-81353f0bb282 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.474049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb638d-2ff5-4131-a958-81353f0bb282 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.476480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb638d-2ff5-4131-a958-81353f0bb282 None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.478223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb638d-2ff5-4131-a958-81353f0bb282 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.480162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 180/345] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.481261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb638d-2ff5-4131-a958-81353f0bb282 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.523991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.577390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 166/346] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:53.579176 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cad019-ce72-41dc-a5b9-cf1d415b490e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.636173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb638d-2ff5-4131-a958-81353f0bb282 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.675727 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.732407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22ec24-7a9e-489a-a947-5aebc86d70e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.732794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22ec24-7a9e-489a-a947-5aebc86d70e5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.733069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22ec24-7a9e-489a-a947-5aebc86d70e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.733341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22ec24-7a9e-489a-a947-5aebc86d70e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.736424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cad019-ce72-41dc-a5b9-cf1d415b490e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.786003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0151a-f75c-4586-ae98-33f096646c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.786658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0151a-f75c-4586-ae98-33f096646c40 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.786961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0151a-f75c-4586-ae98-33f096646c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.787237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0151a-f75c-4586-ae98-33f096646c40 None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.808398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0151a-f75c-4586-ae98-33f096646c40 None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.809889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0151a-f75c-4586-ae98-33f096646c40 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.811647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 167/347] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.817553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0151a-f75c-4586-ae98-33f096646c40 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.878939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22ec24-7a9e-489a-a947-5aebc86d70e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.880394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22ec24-7a9e-489a-a947-5aebc86d70e5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.948769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.013085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0151a-f75c-4586-ae98-33f096646c40 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.034904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.056547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 181/348] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:54.057739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22ec24-7a9e-489a-a947-5aebc86d70e5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.059204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c7acf-1b8e-48fc-9ebe-057ea9c994d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.059883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7acf-1b8e-48fc-9ebe-057ea9c994d7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.060170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7acf-1b8e-48fc-9ebe-057ea9c994d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.060448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7acf-1b8e-48fc-9ebe-057ea9c994d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.061185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c7acf-1b8e-48fc-9ebe-057ea9c994d7 None admin] RBAC: Adding query filter params (group.id=ffeedce6d78742afac2b704ef77ab9f2, role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:54.061496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c7acf-1b8e-48fc-9ebe-057ea9c994d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.063124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c7acf-1b8e-48fc-9ebe-057ea9c994d7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.074288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 168/349] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/role_assignments?group.id=ffeedce6d78742afac2b704ef77ab9f2&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.075690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdae0d5-8879-47c6-96c6-8c12abef0ea6 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.075981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdae0d5-8879-47c6-96c6-8c12abef0ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.076259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdae0d5-8879-47c6-96c6-8c12abef0ea6 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.077636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 169/350] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:54.078274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdae0d5-8879-47c6-96c6-8c12abef0ea6 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.176571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22ec24-7a9e-489a-a947-5aebc86d70e5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.208655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.216771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70170178-03f6-4ca2-8d7d-22b348e5ab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.217700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70170178-03f6-4ca2-8d7d-22b348e5ab6a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.218202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70170178-03f6-4ca2-8d7d-22b348e5ab6a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.218676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70170178-03f6-4ca2-8d7d-22b348e5ab6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.228563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdae0d5-8879-47c6-96c6-8c12abef0ea6 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.232962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70170178-03f6-4ca2-8d7d-22b348e5ab6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-70170178-03f6-4ca2-8d7d-22b348e5ab6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:54.238697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:54.239712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:54.240539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:54.244489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 182/351] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:54.246063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703562a5-cc05-4edf-bf16-1ccba3c97365 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.247033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703562a5-cc05-4edf-bf16-1ccba3c97365 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.247694 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703562a5-cc05-4edf-bf16-1ccba3c97365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.269189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-703562a5-cc05-4edf-bf16-1ccba3c97365 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:54.270805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-703562a5-cc05-4edf-bf16-1ccba3c97365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:54.294471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083b974-cf09-4d2b-913a-c3b1f703fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.295178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083b974-cf09-4d2b-913a-c3b1f703fc2f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.295707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083b974-cf09-4d2b-913a-c3b1f703fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.295707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083b974-cf09-4d2b-913a-c3b1f703fc2f None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.296921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083b974-cf09-4d2b-913a-c3b1f703fc2f None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.298394 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083b974-cf09-4d2b-913a-c3b1f703fc2f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.300130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 170/352] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.305672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083b974-cf09-4d2b-913a-c3b1f703fc2f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.366559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 183/353] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:54.371886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703562a5-cc05-4edf-bf16-1ccba3c97365 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.386595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.498150 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083b974-cf09-4d2b-913a-c3b1f703fc2f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.531579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703562a5-cc05-4edf-bf16-1ccba3c97365 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.545430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01055e20-2117-42e5-ab89-67ed839e55f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.546369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01055e20-2117-42e5-ab89-67ed839e55f3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.546920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01055e20-2117-42e5-ab89-67ed839e55f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.547399 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01055e20-2117-42e5-ab89-67ed839e55f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.548271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01055e20-2117-42e5-ab89-67ed839e55f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:54.548773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01055e20-2117-42e5-ab89-67ed839e55f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.554476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01055e20-2117-42e5-ab89-67ed839e55f3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.554630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.565446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 171/354] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.566524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d5eb-5580-47af-b3eb-23fb519bbcbb None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.566810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d5eb-5580-47af-b3eb-23fb519bbcbb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.567171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d5eb-5580-47af-b3eb-23fb519bbcbb None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.568492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 172/355] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:54.572534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff819154-c171-493e-87c0-98dd65a43a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.573190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff819154-c171-493e-87c0-98dd65a43a65 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.573471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff819154-c171-493e-87c0-98dd65a43a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.573835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc17081-5b8e-493c-a309-78854b174c5f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.573941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff819154-c171-493e-87c0-98dd65a43a65 None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.574017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc17081-5b8e-493c-a309-78854b174c5f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.574243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc17081-5b8e-493c-a309-78854b174c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.575072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff819154-c171-493e-87c0-98dd65a43a65 None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.576505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff819154-c171-493e-87c0-98dd65a43a65 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.578340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 184/356] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.579317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff819154-c171-493e-87c0-98dd65a43a65 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.595948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6cc17081-5b8e-493c-a309-78854b174c5f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:54.597161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6cc17081-5b8e-493c-a309-78854b174c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:54.675103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 173/357] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:54.675935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc17081-5b8e-493c-a309-78854b174c5f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.723466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.800593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff819154-c171-493e-87c0-98dd65a43a65 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.809250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc17081-5b8e-493c-a309-78854b174c5f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.843053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07b380-d820-426d-b459-04a89db113c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.843671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b380-d820-426d-b459-04a89db113c8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.844282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b380-d820-426d-b459-04a89db113c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.844396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07b380-d820-426d-b459-04a89db113c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.863555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.864176 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.865880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.866159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.866944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] RBAC: Adding query filter params (group.id=ffeedce6d78742afac2b704ef77ab9f2, role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:54.867259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.868694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.881757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 174/358] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/role_assignments?group.id=ffeedce6d78742afac2b704ef77ab9f2&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=2e40648646094078acac22f8eb94dd98 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.884629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.965235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.992892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07b380-d820-426d-b459-04a89db113c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.994389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07b380-d820-426d-b459-04a89db113c8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:55.049335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8a931-01d3-4918-a146-031b04e2f764 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:55.067354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:55.129792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 185/359] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:55.131340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469da3d-4eb7-4035-8686-16800722f435 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.131639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469da3d-4eb7-4035-8686-16800722f435 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.132504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469da3d-4eb7-4035-8686-16800722f435 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.134189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 186/360] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:55.135304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469da3d-4eb7-4035-8686-16800722f435 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:55.151715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeea1ab-803e-4543-bf45-c4cf847f4908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:55.152376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea1ab-803e-4543-bf45-c4cf847f4908 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.152662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea1ab-803e-4543-bf45-c4cf847f4908 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.152935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea1ab-803e-4543-bf45-c4cf847f4908 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.187667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeea1ab-803e-4543-bf45-c4cf847f4908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-0eeea1ab-803e-4543-bf45-c4cf847f4908 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:55.193333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:55.195009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:55.196024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 175/361] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:55.196792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f46344-7f88-4969-9d41-e5d4dd44a859 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.197053 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f46344-7f88-4969-9d41-e5d4dd44a859 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.197326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f46344-7f88-4969-9d41-e5d4dd44a859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.232923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-15f46344-7f88-4969-9d41-e5d4dd44a859 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:55.234482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-15f46344-7f88-4969-9d41-e5d4dd44a859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:55.241966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:55.263698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469da3d-4eb7-4035-8686-16800722f435 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:55.309174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1cf6-5ea3-4dc5-972f-c6b60809ea35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:55.309945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1cf6-5ea3-4dc5-972f-c6b60809ea35 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.310295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1cf6-5ea3-4dc5-972f-c6b60809ea35 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.310675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1cf6-5ea3-4dc5-972f-c6b60809ea35 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.312002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff1cf6-5ea3-4dc5-972f-c6b60809ea35 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:55.313618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff1cf6-5ea3-4dc5-972f-c6b60809ea35 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:55.319189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 187/362] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:55 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:55.320634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1cf6-5ea3-4dc5-972f-c6b60809ea35 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:55.330059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 176/363] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:55.331035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f46344-7f88-4969-9d41-e5d4dd44a859 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:55.458906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:55.459627 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1cf6-5ea3-4dc5-972f-c6b60809ea35 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:55.469355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f46344-7f88-4969-9d41-e5d4dd44a859 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:55.511483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceafd4c-e2c9-4599-be02-553e605528f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:55.512379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafd4c-e2c9-4599-be02-553e605528f3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.512878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafd4c-e2c9-4599-be02-553e605528f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.513231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceafd4c-e2c9-4599-be02-553e605528f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.526246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceafd4c-e2c9-4599-be02-553e605528f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-4ceafd4c-e2c9-4599-be02-553e605528f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:55.532378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:55.533454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:55.534435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:55.536196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 177/364] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:55.539290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceafd4c-e2c9-4599-be02-553e605528f3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:55.555735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:55.556896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.557239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.557716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.558584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:55.558969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:55.560409 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:55.578277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 188/365] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:55.579362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:55.627984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:55.671735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceafd4c-e2c9-4599-be02-553e605528f3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:55.735318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9748d62-9c7f-47f0-8c0d-a0da39e59ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:55.735970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9748d62-9c7f-47f0-8c0d-a0da39e59ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.736250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9748d62-9c7f-47f0-8c0d-a0da39e59ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.736527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9748d62-9c7f-47f0-8c0d-a0da39e59ab3 None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.737654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9748d62-9c7f-47f0-8c0d-a0da39e59ab3 None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:55.739157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9748d62-9c7f-47f0-8c0d-a0da39e59ab3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:55.740736 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 178/366] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:45:55 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:55.741780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9748d62-9c7f-47f0-8c0d-a0da39e59ab3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:55.779274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:55.785342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25cd57-45d5-4276-8c87-c8e463d58c0d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:55.832536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:55.833068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.833351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.833624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.834293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:55.834594 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:55.836021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:55.850473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 189/367] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:55.851758 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:55.868467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9748d62-9c7f-47f0-8c0d-a0da39e59ab3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:55.914356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06f9f9-ffe0-4e5d-858e-fc7c8fd209a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:55.915010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f9f9-ffe0-4e5d-858e-fc7c8fd209a1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.915294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f9f9-ffe0-4e5d-858e-fc7c8fd209a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.915566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f9f9-ffe0-4e5d-858e-fc7c8fd209a1 None admin] PATH_INFO: `/v3/users/9fb20fb6dd774623a52a3263e7f63309` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.917416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06f9f9-ffe0-4e5d-858e-fc7c8fd209a1 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb20fb6dd774623a52a3263e7f63309)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:55.919261 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06f9f9-ffe0-4e5d-858e-fc7c8fd209a1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:55.921556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 179/368] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:55 2022] GET /identity/v3/users/9fb20fb6dd774623a52a3263e7f63309 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:55.922212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06f9f9-ffe0-4e5d-858e-fc7c8fd209a1 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:55.959500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:55.982013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c94be-0019-458a-b170-0a21b22c37bb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.021978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caaacaa-df0b-4c53-83a2-f20499804b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.023514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaacaa-df0b-4c53-83a2-f20499804b76 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.023807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaacaa-df0b-4c53-83a2-f20499804b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.024124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaacaa-df0b-4c53-83a2-f20499804b76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.037125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caaacaa-df0b-4c53-83a2-f20499804b76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3caaacaa-df0b-4c53-83a2-f20499804b76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:56.043427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:56.045489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:56.046789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:56.048107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.048236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 190/369] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:56.048236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caaacaa-df0b-4c53-83a2-f20499804b76 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.058558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06f9f9-ffe0-4e5d-858e-fc7c8fd209a1 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.104942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.105752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.106092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.106371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.107583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] RBAC: Adding query filter params (group.id=ffeedce6d78742afac2b704ef77ab9f2, role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:56.107895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.109334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.117595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 180/370] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:45:55 2022] GET /identity/v3/role_assignments?group.id=ffeedce6d78742afac2b704ef77ab9f2&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:56.118434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.200705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caaacaa-df0b-4c53-83a2-f20499804b76 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.248429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcda24-a035-44d0-83c9-e236e6540360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.249075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcda24-a035-44d0-83c9-e236e6540360 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.249357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcda24-a035-44d0-83c9-e236e6540360 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.249631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcda24-a035-44d0-83c9-e236e6540360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.265419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.343271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcda24-a035-44d0-83c9-e236e6540360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.344858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcda24-a035-44d0-83c9-e236e6540360 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.365401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebee1e-b0f7-44fb-92a2-5155208a67a2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.435678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.436287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.436562 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.436830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.437627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] RBAC: Adding query filter params (role.id=8aad1777fec94e0b8c3dc239ae4864c6, user.id=9fb20fb6dd774623a52a3263e7f63309, scope.system=all) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:56.437939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.439679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.462658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 181/371] 10.222.0.52 () {56 vars in 1308 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/role_assignments?user.id=9fb20fb6dd774623a52a3263e7f63309&scope.system=all&role.id=8aad1777fec94e0b8c3dc239ae4864c6 => generated 510 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:56.463532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.470031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 191/372] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:56.477328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcda24-a035-44d0-83c9-e236e6540360 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.491967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.545795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.628838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcda24-a035-44d0-83c9-e236e6540360 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.629551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd35d8-0044-4c87-a131-dd4e21bfba4e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.679801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.680512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.680789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.681059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.681833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:56.682011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8e0b-32c0-4a44-8c01-a32a5f96dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.682174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.682497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8e0b-32c0-4a44-8c01-a32a5f96dbde None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.682778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8e0b-32c0-4a44-8c01-a32a5f96dbde None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.683140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8e0b-32c0-4a44-8c01-a32a5f96dbde None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.683575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.684345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb8e0b-32c0-4a44-8c01-a32a5f96dbde None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.685995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb8e0b-32c0-4a44-8c01-a32a5f96dbde None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.691483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 192/373] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:56.693309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 182/374] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:56.695131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8e0b-32c0-4a44-8c01-a32a5f96dbde None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.700890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.818183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.853339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8e0b-32c0-4a44-8c01-a32a5f96dbde None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.868669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.873559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326096b-06d3-4853-82b7-aff9e2988a0c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.918268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902098c-f1cf-4722-90a7-73fb79deff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.918840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902098c-f1cf-4722-90a7-73fb79deff04 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.919290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902098c-f1cf-4722-90a7-73fb79deff04 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.919569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902098c-f1cf-4722-90a7-73fb79deff04 None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.921472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902098c-f1cf-4722-90a7-73fb79deff04 None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.922917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902098c-f1cf-4722-90a7-73fb79deff04 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.929495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 193/375] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:56.939985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90a010-e09f-4199-8fbc-288a74420319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.940603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90a010-e09f-4199-8fbc-288a74420319 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.940886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90a010-e09f-4199-8fbc-288a74420319 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.941162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90a010-e09f-4199-8fbc-288a74420319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.946314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902098c-f1cf-4722-90a7-73fb79deff04 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:57.044352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90a010-e09f-4199-8fbc-288a74420319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.045821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90a010-e09f-4199-8fbc-288a74420319 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.105454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902098c-f1cf-4722-90a7-73fb79deff04 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.136509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:57.162421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.163639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.163921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.164191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.164931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] RBAC: Adding query filter params (group.id=ffeedce6d78742afac2b704ef77ab9f2, role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:57.165234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.166882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.173227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 194/376] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/role_assignments?group.id=ffeedce6d78742afac2b704ef77ab9f2&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=2e40648646094078acac22f8eb94dd98 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:57.183165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 183/377] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:57.184616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:57.265343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:57.339281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f09e02-2da3-4abc-85cc-5091d4d87a6c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.385340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96239ea0-80dc-4033-8a81-783bd46bc6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.386002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96239ea0-80dc-4033-8a81-783bd46bc6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.386284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96239ea0-80dc-4033-8a81-783bd46bc6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.386552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96239ea0-80dc-4033-8a81-783bd46bc6c1 None admin] PATH_INFO: `/v3/projects/992fcf5a85d6497db94cb36327aec1f0` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.387785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96239ea0-80dc-4033-8a81-783bd46bc6c1 None admin] RBAC: Authorizing `identity:get_project(project_id=992fcf5a85d6497db94cb36327aec1f0)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.389344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96239ea0-80dc-4033-8a81-783bd46bc6c1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.391354 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 195/378] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:57 2022] GET /identity/v3/projects/992fcf5a85d6497db94cb36327aec1f0 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:57.396878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90a010-e09f-4199-8fbc-288a74420319 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:57.494715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e0da-df6d-4c1d-82e8-0b8bb0d70ea3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.494921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e0da-df6d-4c1d-82e8-0b8bb0d70ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.495198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e0da-df6d-4c1d-82e8-0b8bb0d70ea3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.497518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 196/379] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:57.504199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4091cfb-39e1-469f-a16f-42c607eb0cd1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.504433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4091cfb-39e1-469f-a16f-42c607eb0cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.504678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4091cfb-39e1-469f-a16f-42c607eb0cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.536322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b4091cfb-39e1-469f-a16f-42c607eb0cd1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:57.537816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b4091cfb-39e1-469f-a16f-42c607eb0cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:57.559947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90a010-e09f-4199-8fbc-288a74420319 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.612076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f911a6-8c39-4e14-96fa-bb990c27b28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.613248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f911a6-8c39-4e14-96fa-bb990c27b28d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.613979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f911a6-8c39-4e14-96fa-bb990c27b28d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.614433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f911a6-8c39-4e14-96fa-bb990c27b28d None admin] PATH_INFO: `/v3/groups/b9eb1033fcf6474eb12163421df03e40` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.621669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f911a6-8c39-4e14-96fa-bb990c27b28d None admin] RBAC: Authorizing `identity:get_group(group_id=b9eb1033fcf6474eb12163421df03e40)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.627939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f911a6-8c39-4e14-96fa-bb990c27b28d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.629866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 184/380] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:45:57 2022] GET /identity/v3/groups/b9eb1033fcf6474eb12163421df03e40 => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:57.638237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f911a6-8c39-4e14-96fa-bb990c27b28d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:57.652708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 197/381] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:57.690491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:57.756447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f911a6-8c39-4e14-96fa-bb990c27b28d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.792106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34653b-5417-4740-b1df-b6ed845b5318 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.792424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34653b-5417-4740-b1df-b6ed845b5318 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.792663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34653b-5417-4740-b1df-b6ed845b5318 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.794061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 198/382] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:57.799191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a9d6-7246-41bf-89df-1d8530f51630 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.799710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a9d6-7246-41bf-89df-1d8530f51630 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.800178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a9d6-7246-41bf-89df-1d8530f51630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.803341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.804520 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.804804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.805071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.809188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] RBAC: Adding query filter params (group.id=b9eb1033fcf6474eb12163421df03e40, role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=992fcf5a85d6497db94cb36327aec1f0) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:57.809188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.809188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.814204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 185/383] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:45:57 2022] GET /identity/v3/role_assignments?group.id=b9eb1033fcf6474eb12163421df03e40&scope.project.id=992fcf5a85d6497db94cb36327aec1f0&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:57.840134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0ca1a9d6-7246-41bf-89df-1d8530f51630 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:57.841628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0ca1a9d6-7246-41bf-89df-1d8530f51630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:57.886109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:57.940926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 199/384] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:58.087654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.228315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee9a65-61e8-4195-86e5-459830e7ee44 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.229862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.272604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd968aa-44d9-484f-ab1a-e1df3a88d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.273235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd968aa-44d9-484f-ab1a-e1df3a88d62d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.273502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd968aa-44d9-484f-ab1a-e1df3a88d62d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.273773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd968aa-44d9-484f-ab1a-e1df3a88d62d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.287438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd968aa-44d9-484f-ab1a-e1df3a88d62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:58.288010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a9d6-7246-41bf-89df-1d8530f51630 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-4cd968aa-44d9-484f-ab1a-e1df3a88d62d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:58.293412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:58.294294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:45:58.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 186/385] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:58.302051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd968aa-44d9-484f-ab1a-e1df3a88d62d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.468549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd968aa-44d9-484f-ab1a-e1df3a88d62d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.482534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a9d6-7246-41bf-89df-1d8530f51630 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.520473 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.521082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.521363 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.521634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.522300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:58.523201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.524591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.539547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 187/386] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:58.547905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.563291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb7c80-5580-434c-b81c-986cc0d39a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.564193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb7c80-5580-434c-b81c-986cc0d39a3b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.564467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb7c80-5580-434c-b81c-986cc0d39a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.564728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb7c80-5580-434c-b81c-986cc0d39a3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.583523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb7c80-5580-434c-b81c-986cc0d39a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-35bb7c80-5580-434c-b81c-986cc0d39a3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:58.589439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:58.590313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:58.591145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 200/387] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:58.592321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.599550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb7c80-5580-434c-b81c-986cc0d39a3b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.659560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.715372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9189831-80f7-4b60-a4f4-1fd35724dc78 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.772999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb7c80-5580-434c-b81c-986cc0d39a3b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.807614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611ce50-00fa-4df3-87bb-2ff6f8cdac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.808276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611ce50-00fa-4df3-87bb-2ff6f8cdac33 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.808566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611ce50-00fa-4df3-87bb-2ff6f8cdac33 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.808849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611ce50-00fa-4df3-87bb-2ff6f8cdac33 None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.809997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611ce50-00fa-4df3-87bb-2ff6f8cdac33 None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.810546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.811186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.811467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.811576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611ce50-00fa-4df3-87bb-2ff6f8cdac33 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.811731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.812668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:58.813103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.814301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 188/388] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:58.815459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.820084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611ce50-00fa-4df3-87bb-2ff6f8cdac33 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.825821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 201/389] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:58.831028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.938421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.947563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8d1e7-4d34-4af5-8584-b0e067e397a1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.992198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b280140-1fbb-4852-af8d-770e4f223231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.993073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b280140-1fbb-4852-af8d-770e4f223231 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.993514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b280140-1fbb-4852-af8d-770e4f223231 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.993918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b280140-1fbb-4852-af8d-770e4f223231 None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.995403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b280140-1fbb-4852-af8d-770e4f223231 None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.996958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b280140-1fbb-4852-af8d-770e4f223231 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.998668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 202/390] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:59.004624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b280140-1fbb-4852-af8d-770e4f223231 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:59.078600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:59.090418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611ce50-00fa-4df3-87bb-2ff6f8cdac33 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:59.128652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beb202-f574-47e3-9d41-0d75bf357cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:59.129708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb202-f574-47e3-9d41-0d75bf357cbf None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.130179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb202-f574-47e3-9d41-0d75bf357cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.130673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb202-f574-47e3-9d41-0d75bf357cbf None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.132147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beb202-f574-47e3-9d41-0d75bf357cbf None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:59.133872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beb202-f574-47e3-9d41-0d75bf357cbf None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:59.136054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 189/391] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:59.137428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc818b65-dc86-45de-ad13-4aa504fd0f44 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.137906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc818b65-dc86-45de-ad13-4aa504fd0f44 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.138466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc818b65-dc86-45de-ad13-4aa504fd0f44 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.139908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 190/392] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:59.149642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7aacf-ef34-4b77-a958-cc6a50f4cd71 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.150943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7aacf-ef34-4b77-a958-cc6a50f4cd71 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.151601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7aacf-ef34-4b77-a958-cc6a50f4cd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.160920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b280140-1fbb-4852-af8d-770e4f223231 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:59.172837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-74b7aacf-ef34-4b77-a958-cc6a50f4cd71 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:59.174297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-74b7aacf-ef34-4b77-a958-cc6a50f4cd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:59.202397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc02ea-453d-4095-b0c3-ffbc6078b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:59.202976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc02ea-453d-4095-b0c3-ffbc6078b9de None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.203260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc02ea-453d-4095-b0c3-ffbc6078b9de None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.203531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc02ea-453d-4095-b0c3-ffbc6078b9de None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.204679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc02ea-453d-4095-b0c3-ffbc6078b9de None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:59.206099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc02ea-453d-4095-b0c3-ffbc6078b9de None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:59.208085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 203/393] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:59 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:59.208926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc02ea-453d-4095-b0c3-ffbc6078b9de None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:59.244698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:59.254151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 191/394] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:59.255789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7aacf-ef34-4b77-a958-cc6a50f4cd71 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:59.350055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc02ea-453d-4095-b0c3-ffbc6078b9de None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:59.372607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:59.384470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7aacf-ef34-4b77-a958-cc6a50f4cd71 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:59.385896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84327081-c71b-4b9e-b267-012212f48f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:59.386806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327081-c71b-4b9e-b267-012212f48f62 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.387126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327081-c71b-4b9e-b267-012212f48f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.387400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327081-c71b-4b9e-b267-012212f48f62 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/groups/ffeedce6d78742afac2b704ef77ab9f2/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.390176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84327081-c71b-4b9e-b267-012212f48f62 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bf2c16b2e8b435bbd4cf6885ae631db, group_id=ffeedce6d78742afac2b704ef77ab9f2, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:59.391633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84327081-c71b-4b9e-b267-012212f48f62 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:59.416141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 204/395] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Oct 10 11:45:59 2022] PUT /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/groups/ffeedce6d78742afac2b704ef77ab9f2/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:59.439886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f50cad-7399-4d6c-9a9c-8ef4beab2a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:59.440999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50cad-7399-4d6c-9a9c-8ef4beab2a72 None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.441649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50cad-7399-4d6c-9a9c-8ef4beab2a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.442115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f50cad-7399-4d6c-9a9c-8ef4beab2a72 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/groups/ffeedce6d78742afac2b704ef77ab9f2/roles/2e40648646094078acac22f8eb94dd98` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.445124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f50cad-7399-4d6c-9a9c-8ef4beab2a72 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bf2c16b2e8b435bbd4cf6885ae631db, group_id=ffeedce6d78742afac2b704ef77ab9f2, role_id=2e40648646094078acac22f8eb94dd98)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:59.447512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f50cad-7399-4d6c-9a9c-8ef4beab2a72 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:59.465146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 192/396] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Oct 10 11:45:59 2022] PUT /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db/groups/ffeedce6d78742afac2b704ef77ab9f2/roles/2e40648646094078acac22f8eb94dd98 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:59.689481 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84327081-c71b-4b9e-b267-012212f48f62 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:59.784022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:59.801885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84327081-c71b-4b9e-b267-012212f48f62 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:59.842002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fcca1-1fa7-4ff5-b8c2-7d17ebe7cfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:59.842652 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fcca1-1fa7-4ff5-b8c2-7d17ebe7cfd6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.842957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fcca1-1fa7-4ff5-b8c2-7d17ebe7cfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.843221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fcca1-1fa7-4ff5-b8c2-7d17ebe7cfd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.854943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fcca1-1fa7-4ff5-b8c2-7d17ebe7cfd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-754fcca1-1fa7-4ff5-b8c2-7d17ebe7cfd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:59.860090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:59.861043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:45:59.861832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 205/397] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:59.867110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f50cad-7399-4d6c-9a9c-8ef4beab2a72 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.025739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f50cad-7399-4d6c-9a9c-8ef4beab2a72 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:00.065366 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:00.065901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.066180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.066451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.067184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:00.067491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:00.068855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:00.082662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 193/398] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:00.088012 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fcca1-1fa7-4ff5-b8c2-7d17ebe7cfd6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.217850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fcca1-1fa7-4ff5-b8c2-7d17ebe7cfd6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:00.262309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718abd5-87e8-467f-accd-711d7210078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:00.262969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718abd5-87e8-467f-accd-711d7210078b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.263247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718abd5-87e8-467f-accd-711d7210078b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.263517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718abd5-87e8-467f-accd-711d7210078b None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.264697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718abd5-87e8-467f-accd-711d7210078b None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:00.266107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718abd5-87e8-467f-accd-711d7210078b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:00.268018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 206/399] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:00 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:00.273899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.305391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:00.431461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3b5a0-f1ad-46d8-a3c4-d4a46c5834c4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:00.484577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3430f8-256b-43f8-9fb7-95fb482e0321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:00.485581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3430f8-256b-43f8-9fb7-95fb482e0321 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.486059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3430f8-256b-43f8-9fb7-95fb482e0321 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.486528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3430f8-256b-43f8-9fb7-95fb482e0321 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.488604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3430f8-256b-43f8-9fb7-95fb482e0321 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:00.490306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3430f8-256b-43f8-9fb7-95fb482e0321 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:00.493033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 194/400] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:00 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:00.498987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718abd5-87e8-467f-accd-711d7210078b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.517213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:00.534362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973705d-4176-4885-8064-18e8170408ec None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.534685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973705d-4176-4885-8064-18e8170408ec None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.534972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973705d-4176-4885-8064-18e8170408ec None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.536600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 195/401] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:00.545103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7b24c-47c0-4c2c-a6c9-d07fe05e4ae1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.545388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7b24c-47c0-4c2c-a6c9-d07fe05e4ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.545663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7b24c-47c0-4c2c-a6c9-d07fe05e4ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.575794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-41b7b24c-47c0-4c2c-a6c9-d07fe05e4ae1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:00.577071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-41b7b24c-47c0-4c2c-a6c9-d07fe05e4ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:00.619900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718abd5-87e8-467f-accd-711d7210078b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:00.671842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834147e-a737-40f8-a0ac-f2a48280a557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:00.672892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834147e-a737-40f8-a0ac-f2a48280a557 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.673397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834147e-a737-40f8-a0ac-f2a48280a557 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.673853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834147e-a737-40f8-a0ac-f2a48280a557 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/groups/ffeedce6d78742afac2b704ef77ab9f2/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.676240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834147e-a737-40f8-a0ac-f2a48280a557 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcc729b56f9c4ffb9e49f733ba16d591, group_id=ffeedce6d78742afac2b704ef77ab9f2, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:00.677896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834147e-a737-40f8-a0ac-f2a48280a557 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:00.687011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 196/402] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:00.705443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 207/403] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Oct 10 11:46:00 2022] PUT /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/groups/ffeedce6d78742afac2b704ef77ab9f2/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:00.844435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc965e-b05b-4292-88cd-b614d74c83e8 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.844761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc965e-b05b-4292-88cd-b614d74c83e8 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.844987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc965e-b05b-4292-88cd-b614d74c83e8 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.846368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 197/404] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:00.853357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42b3a9-0fbc-4bba-bb82-b9c0419f3e5c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.853988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42b3a9-0fbc-4bba-bb82-b9c0419f3e5c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.854462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42b3a9-0fbc-4bba-bb82-b9c0419f3e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.883991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bf42b3a9-0fbc-4bba-bb82-b9c0419f3e5c None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:00.885595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bf42b3a9-0fbc-4bba-bb82-b9c0419f3e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:00.964414 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 208/405] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:01.038602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc965e-b05b-4292-88cd-b614d74c83e8 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.052599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.157758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc965e-b05b-4292-88cd-b614d74c83e8 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.207479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02314e23-97e4-4d3c-8105-8910caa161e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.208456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02314e23-97e4-4d3c-8105-8910caa161e9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.208965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02314e23-97e4-4d3c-8105-8910caa161e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.209435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02314e23-97e4-4d3c-8105-8910caa161e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.226353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02314e23-97e4-4d3c-8105-8910caa161e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-02314e23-97e4-4d3c-8105-8910caa161e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:01.231692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:01.232539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:01.233315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:01.238975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 198/406] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:01.242667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42b3a9-0fbc-4bba-bb82-b9c0419f3e5c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.247261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.373844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42b3a9-0fbc-4bba-bb82-b9c0419f3e5c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.398410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02314e23-97e4-4d3c-8105-8910caa161e9 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.427588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.428218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.429020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.429306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:01.430279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.431663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:01.448165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 209/407] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:01.453913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.525132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.559414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.596294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa71a2b-303e-43e3-8c56-51a4780c5621 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.619648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02314e23-97e4-4d3c-8105-8910caa161e9 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.668191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d750663-0a64-42c4-bc33-9efc1dd00e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.668831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d750663-0a64-42c4-bc33-9efc1dd00e37 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.669113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d750663-0a64-42c4-bc33-9efc1dd00e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.669390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d750663-0a64-42c4-bc33-9efc1dd00e37 None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.671608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d750663-0a64-42c4-bc33-9efc1dd00e37 None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.673040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d750663-0a64-42c4-bc33-9efc1dd00e37 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:01.682603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 210/408] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:01.685726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d750663-0a64-42c4-bc33-9efc1dd00e37 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.699791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b99c1-95b6-47bd-b685-63c9d68437a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.700435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b99c1-95b6-47bd-b685-63c9d68437a3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.700722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b99c1-95b6-47bd-b685-63c9d68437a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.700998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b99c1-95b6-47bd-b685-63c9d68437a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.732248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b99c1-95b6-47bd-b685-63c9d68437a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b12b99c1-95b6-47bd-b685-63c9d68437a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:01.749698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:01.750617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:01.751443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 199/409] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:01.759819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b99c1-95b6-47bd-b685-63c9d68437a3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.813240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d750663-0a64-42c4-bc33-9efc1dd00e37 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.867631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9adc1e-959d-4a4c-831e-24f89a56bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.868286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9adc1e-959d-4a4c-831e-24f89a56bec7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.868571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9adc1e-959d-4a4c-831e-24f89a56bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.868849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9adc1e-959d-4a4c-831e-24f89a56bec7 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.870036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9adc1e-959d-4a4c-831e-24f89a56bec7 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.871533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9adc1e-959d-4a4c-831e-24f89a56bec7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:01.874441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 211/410] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:01.879170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9adc1e-959d-4a4c-831e-24f89a56bec7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.885425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.907353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b99c1-95b6-47bd-b685-63c9d68437a3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.970212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.983482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.984387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.984865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.985351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.986225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:01.986724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.988396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:02.006345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 200/411] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:02.011381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.020118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9adc1e-959d-4a4c-831e-24f89a56bec7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.055655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577ebc-7a4c-4ab3-a28f-2cf5a8246372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.056323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577ebc-7a4c-4ab3-a28f-2cf5a8246372 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.056607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577ebc-7a4c-4ab3-a28f-2cf5a8246372 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.056882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577ebc-7a4c-4ab3-a28f-2cf5a8246372 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/groups/ffeedce6d78742afac2b704ef77ab9f2/roles/2e40648646094078acac22f8eb94dd98` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.061028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d577ebc-7a4c-4ab3-a28f-2cf5a8246372 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcc729b56f9c4ffb9e49f733ba16d591, group_id=ffeedce6d78742afac2b704ef77ab9f2, role_id=2e40648646094078acac22f8eb94dd98)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:02.062448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d577ebc-7a4c-4ab3-a28f-2cf5a8246372 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:02.090258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 212/412] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Oct 10 11:46:01 2022] PUT /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591/groups/ffeedce6d78742afac2b704ef77ab9f2/roles/2e40648646094078acac22f8eb94dd98 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:02.196030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac776372-bdd9-4691-b0b8-aa2ef2c12b76 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.243845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58594c33-6fcb-449d-be75-f27a93be6168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.244488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58594c33-6fcb-449d-be75-f27a93be6168 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.244770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58594c33-6fcb-449d-be75-f27a93be6168 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.245045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58594c33-6fcb-449d-be75-f27a93be6168 None admin] PATH_INFO: `/v3/groups/b9eb1033fcf6474eb12163421df03e40` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.246245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58594c33-6fcb-449d-be75-f27a93be6168 None admin] RBAC: Authorizing `identity:get_group(group_id=b9eb1033fcf6474eb12163421df03e40)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:02.247727 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58594c33-6fcb-449d-be75-f27a93be6168 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:02.249671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 201/413] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:02 2022] GET /identity/v3/groups/b9eb1033fcf6474eb12163421df03e40 => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:02.256333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577ebc-7a4c-4ab3-a28f-2cf5a8246372 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.327828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f4e2-9d3e-4bc2-82dc-eddbccb9fa43 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.328350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f4e2-9d3e-4bc2-82dc-eddbccb9fa43 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.328765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f4e2-9d3e-4bc2-82dc-eddbccb9fa43 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.330616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 202/414] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:02.336954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392c842-bb03-4199-aa84-a1efb015f4c1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.337451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392c842-bb03-4199-aa84-a1efb015f4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.337864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392c842-bb03-4199-aa84-a1efb015f4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.358280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c392c842-bb03-4199-aa84-a1efb015f4c1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:02.359777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c392c842-bb03-4199-aa84-a1efb015f4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:02.387600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:02.441022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577ebc-7a4c-4ab3-a28f-2cf5a8246372 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.454308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:02.485989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1cc21-5871-4dfe-bb60-643a509e73de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.486826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1cc21-5871-4dfe-bb60-643a509e73de None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.487172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1cc21-5871-4dfe-bb60-643a509e73de None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.487443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1cc21-5871-4dfe-bb60-643a509e73de None admin] PATH_INFO: `/v3/projects/992fcf5a85d6497db94cb36327aec1f0` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.488563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1cc21-5871-4dfe-bb60-643a509e73de None admin] RBAC: Authorizing `identity:get_project(project_id=992fcf5a85d6497db94cb36327aec1f0)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:02.489977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1cc21-5871-4dfe-bb60-643a509e73de None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:02.492296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 213/415] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:02 2022] GET /identity/v3/projects/992fcf5a85d6497db94cb36327aec1f0 => generated 344 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:02.493795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 203/416] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:02.497807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1cc21-5871-4dfe-bb60-643a509e73de None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.554656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b68677-9da8-4d5b-8e06-389259177226 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.554998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b68677-9da8-4d5b-8e06-389259177226 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.555230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b68677-9da8-4d5b-8e06-389259177226 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.556697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 204/417] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:02.561427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecb59e-ada8-437e-aec1-57088c44df80 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.561710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecb59e-ada8-437e-aec1-57088c44df80 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.561980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecb59e-ada8-437e-aec1-57088c44df80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.580543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-8cecb59e-ada8-437e-aec1-57088c44df80 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:02.581719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8cecb59e-ada8-437e-aec1-57088c44df80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:02.612085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1cc21-5871-4dfe-bb60-643a509e73de None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.646071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:02.655079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e46a2-c68a-4535-a3dd-5f688fd21944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.655838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e46a2-c68a-4535-a3dd-5f688fd21944 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.656118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e46a2-c68a-4535-a3dd-5f688fd21944 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.656394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e46a2-c68a-4535-a3dd-5f688fd21944 None admin] PATH_INFO: `/v3/projects/992fcf5a85d6497db94cb36327aec1f0/groups/b9eb1033fcf6474eb12163421df03e40/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.658335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e46a2-c68a-4535-a3dd-5f688fd21944 None admin] RBAC: Authorizing `identity:create_grant(project_id=992fcf5a85d6497db94cb36327aec1f0, group_id=b9eb1033fcf6474eb12163421df03e40, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:02.659799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e46a2-c68a-4535-a3dd-5f688fd21944 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:02.685701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 214/418] 10.222.0.52 () {58 vars in 1442 bytes} [Mon Oct 10 11:46:02 2022] PUT /identity/v3/projects/992fcf5a85d6497db94cb36327aec1f0/groups/b9eb1033fcf6474eb12163421df03e40/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:02.708751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 205/419] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:02.838844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e46a2-c68a-4535-a3dd-5f688fd21944 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.949460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e46a2-c68a-4535-a3dd-5f688fd21944 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.984842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1069288-9a06-440f-9fcf-ebe0645c86d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.985473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1069288-9a06-440f-9fcf-ebe0645c86d2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.985753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1069288-9a06-440f-9fcf-ebe0645c86d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.986022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1069288-9a06-440f-9fcf-ebe0645c86d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.996793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1069288-9a06-440f-9fcf-ebe0645c86d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b1069288-9a06-440f-9fcf-ebe0645c86d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:03.002128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:03.002999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:03.003775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 215/420] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:03.009959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecb59e-ada8-437e-aec1-57088c44df80 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.072875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1069288-9a06-440f-9fcf-ebe0645c86d2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.135064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.181026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecb59e-ada8-437e-aec1-57088c44df80 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.221471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.227606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1069288-9a06-440f-9fcf-ebe0645c86d2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.264014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.265215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.265848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.266457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.267869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] RBAC: Adding query filter params (name=member) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:03.268439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.268627 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803b89-9c0e-431a-b243-5355d5b0b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.269491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803b89-9c0e-431a-b243-5355d5b0b8bd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.270014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803b89-9c0e-431a-b243-5355d5b0b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.270474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803b89-9c0e-431a-b243-5355d5b0b8bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.271061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.289794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8803b89-9c0e-431a-b243-5355d5b0b8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:03.295979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 206/421] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c8803b89-9c0e-431a-b243-5355d5b0b8bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:03.297961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:03.298805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:03.299647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:03.301592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.303332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 216/422] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:03.311744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803b89-9c0e-431a-b243-5355d5b0b8bd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.447776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0421f-395a-45ad-9798-496eb867f771 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.458025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803b89-9c0e-431a-b243-5355d5b0b8bd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.473308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.484413 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c6b3f-04da-4063-8901-e334d015206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.485636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c6b3f-04da-4063-8901-e334d015206f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.486117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c6b3f-04da-4063-8901-e334d015206f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.486585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c6b3f-04da-4063-8901-e334d015206f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.508759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caf704-8819-487d-a786-7c6a138a1699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.509387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caf704-8819-487d-a786-7c6a138a1699 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.509666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caf704-8819-487d-a786-7c6a138a1699 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.510006 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caf704-8819-487d-a786-7c6a138a1699 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.510683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caf704-8819-487d-a786-7c6a138a1699 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:03.511013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caf704-8819-487d-a786-7c6a138a1699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.512975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caf704-8819-487d-a786-7c6a138a1699 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.525175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 217/423] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:03.527652 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c62e-13d9-45c3-af14-49d185c8e342 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.527944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c62e-13d9-45c3-af14-49d185c8e342 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.528220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c62e-13d9-45c3-af14-49d185c8e342 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.534386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 218/424] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:03.535478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23c62e-13d9-45c3-af14-49d185c8e342 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.545302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.569743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c6b3f-04da-4063-8901-e334d015206f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.571557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c6b3f-04da-4063-8901-e334d015206f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.655026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23c62e-13d9-45c3-af14-49d185c8e342 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.681183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 207/425] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:03.682451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed4db-a951-4054-b01b-1bd8d070d1f0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.682743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed4db-a951-4054-b01b-1bd8d070d1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.683047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed4db-a951-4054-b01b-1bd8d070d1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.693931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4b2e9-c7b1-4325-9ba1-615c6bf4cfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.695506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4b2e9-c7b1-4325-9ba1-615c6bf4cfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.696001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4b2e9-c7b1-4325-9ba1-615c6bf4cfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.696456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4b2e9-c7b1-4325-9ba1-615c6bf4cfa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.703215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-4a0ed4db-a951-4054-b01b-1bd8d070d1f0 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:03.704411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4a0ed4db-a951-4054-b01b-1bd8d070d1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:03.740448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.770789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4b2e9-c7b1-4325-9ba1-615c6bf4cfa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.772683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4b2e9-c7b1-4325-9ba1-615c6bf4cfa8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.787047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 208/426] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:03.788460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ed4db-a951-4054-b01b-1bd8d070d1f0 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.810903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.930689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 219/427] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:03.941124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4b2e9-c7b1-4325-9ba1-615c6bf4cfa8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.019590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.020975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ed4db-a951-4054-b01b-1bd8d070d1f0 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.049583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4b2e9-c7b1-4325-9ba1-615c6bf4cfa8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.087773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2839c-b0f3-4b7a-9225-6c95b9e03c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.088857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2839c-b0f3-4b7a-9225-6c95b9e03c89 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.089325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2839c-b0f3-4b7a-9225-6c95b9e03c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.089781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2839c-b0f3-4b7a-9225-6c95b9e03c89 None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.091130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2839c-b0f3-4b7a-9225-6c95b9e03c89 None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.092876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2839c-b0f3-4b7a-9225-6c95b9e03c89 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.094697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 220/428] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:04.100944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2839c-b0f3-4b7a-9225-6c95b9e03c89 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.107598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724eb16-0569-48aa-90be-7c333af8f3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.108257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724eb16-0569-48aa-90be-7c333af8f3fa None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.108540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724eb16-0569-48aa-90be-7c333af8f3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.108814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724eb16-0569-48aa-90be-7c333af8f3fa None admin] PATH_INFO: `/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.116513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724eb16-0569-48aa-90be-7c333af8f3fa None admin] RBAC: Authorizing `identity:get_project(project_id=3bf2c16b2e8b435bbd4cf6885ae631db)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.117926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724eb16-0569-48aa-90be-7c333af8f3fa None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.119738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 209/429] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/projects/3bf2c16b2e8b435bbd4cf6885ae631db => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:04.120359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724eb16-0569-48aa-90be-7c333af8f3fa None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.214350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.220977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2839c-b0f3-4b7a-9225-6c95b9e03c89 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.235433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724eb16-0569-48aa-90be-7c333af8f3fa None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.262342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce017b16-2ad7-42bc-9816-f762cdbaf9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.263439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce017b16-2ad7-42bc-9816-f762cdbaf9bb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.263994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce017b16-2ad7-42bc-9816-f762cdbaf9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.264457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce017b16-2ad7-42bc-9816-f762cdbaf9bb None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.265779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce017b16-2ad7-42bc-9816-f762cdbaf9bb None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.267523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce017b16-2ad7-42bc-9816-f762cdbaf9bb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.269447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 221/430] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:04.275940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce017b16-2ad7-42bc-9816-f762cdbaf9bb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.283010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c5875-7765-47c3-844f-c894e4aac0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.283644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c5875-7765-47c3-844f-c894e4aac0ef None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.283911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c5875-7765-47c3-844f-c894e4aac0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.284185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c5875-7765-47c3-844f-c894e4aac0ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.290926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.300043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c5875-7765-47c3-844f-c894e4aac0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-696c5875-7765-47c3-844f-c894e4aac0ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:04.306669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:04.307547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:04.308340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 210/431] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:04.308979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c5875-7765-47c3-844f-c894e4aac0ef None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.394653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce017b16-2ad7-42bc-9816-f762cdbaf9bb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.437134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dce49-3d96-462b-a523-dfc7bd85de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.438171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dce49-3d96-462b-a523-dfc7bd85de54 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.439030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dce49-3d96-462b-a523-dfc7bd85de54 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.439525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dce49-3d96-462b-a523-dfc7bd85de54 None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.441018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dce49-3d96-462b-a523-dfc7bd85de54 None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.441352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c5875-7765-47c3-844f-c894e4aac0ef None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.443052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dce49-3d96-462b-a523-dfc7bd85de54 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.446946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 222/432] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:04.446946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dce49-3d96-462b-a523-dfc7bd85de54 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.489517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.490287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.490662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.491091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.491925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] RBAC: Adding query filter params (group.id=ffeedce6d78742afac2b704ef77ab9f2, role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:04.492385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.494460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.510933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 211/433] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/role_assignments?group.id=ffeedce6d78742afac2b704ef77ab9f2&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=2e40648646094078acac22f8eb94dd98 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:04.510933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.510933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.559813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.587368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dce49-3d96-462b-a523-dfc7bd85de54 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.626326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616377d-70a5-4460-967f-bcadf98b3620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.627072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616377d-70a5-4460-967f-bcadf98b3620 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.627426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616377d-70a5-4460-967f-bcadf98b3620 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.627766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616377d-70a5-4460-967f-bcadf98b3620 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.628509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616377d-70a5-4460-967f-bcadf98b3620 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:04.628881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616377d-70a5-4460-967f-bcadf98b3620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.630367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616377d-70a5-4460-967f-bcadf98b3620 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.640533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 223/434] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:04.641968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb52386-0df2-44c7-a6e8-652c70eddeb4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.642354 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb52386-0df2-44c7-a6e8-652c70eddeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.642729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb52386-0df2-44c7-a6e8-652c70eddeb4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.644217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 224/435] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:04.649379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb52386-0df2-44c7-a6e8-652c70eddeb4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.653094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89726b89-a0a7-4f66-96ab-b6b9c24a2bb2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.712913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e21841-c582-4036-9d5e-70ba0cf4c3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.713529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e21841-c582-4036-9d5e-70ba0cf4c3ce None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.713809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e21841-c582-4036-9d5e-70ba0cf4c3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.714081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e21841-c582-4036-9d5e-70ba0cf4c3ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.714838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e21841-c582-4036-9d5e-70ba0cf4c3ce None admin] RBAC: Adding query filter params (group.id=ffeedce6d78742afac2b704ef77ab9f2, role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:04.715246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e21841-c582-4036-9d5e-70ba0cf4c3ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.716697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e21841-c582-4036-9d5e-70ba0cf4c3ce None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.724175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 212/436] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/role_assignments?group.id=ffeedce6d78742afac2b704ef77ab9f2&scope.project.id=3bf2c16b2e8b435bbd4cf6885ae631db&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:04.725408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325aceb-2696-402d-bac5-5146a4adc2c2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.725690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325aceb-2696-402d-bac5-5146a4adc2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.725962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325aceb-2696-402d-bac5-5146a4adc2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.746960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-2325aceb-2696-402d-bac5-5146a4adc2c2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:04.748187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2325aceb-2696-402d-bac5-5146a4adc2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:04.796873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb52386-0df2-44c7-a6e8-652c70eddeb4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.836799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.841647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 213/437] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:04.848207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d9d9b-4dcd-492d-9102-4ba58ef08541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.848844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d9d9b-4dcd-492d-9102-4ba58ef08541 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.849129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d9d9b-4dcd-492d-9102-4ba58ef08541 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.849407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d9d9b-4dcd-492d-9102-4ba58ef08541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.863314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.926736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d9d9b-4dcd-492d-9102-4ba58ef08541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.928317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d9d9b-4dcd-492d-9102-4ba58ef08541 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.034857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 225/438] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:05.040833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325aceb-2696-402d-bac5-5146a4adc2c2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.118952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d9d9b-4dcd-492d-9102-4ba58ef08541 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.158502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325aceb-2696-402d-bac5-5146a4adc2c2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.169105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.196793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a184a0-a706-439e-90ec-c2f93853d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.197451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a184a0-a706-439e-90ec-c2f93853d7dd None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.197734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a184a0-a706-439e-90ec-c2f93853d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.198008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a184a0-a706-439e-90ec-c2f93853d7dd None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.199223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a184a0-a706-439e-90ec-c2f93853d7dd None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.200714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a184a0-a706-439e-90ec-c2f93853d7dd None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.207034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 214/439] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:05.209546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a184a0-a706-439e-90ec-c2f93853d7dd None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.248265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d9d9b-4dcd-492d-9102-4ba58ef08541 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.294424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55d771-23d3-4973-abb3-903de845a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.295265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.296020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55d771-23d3-4973-abb3-903de845a908 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.296316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55d771-23d3-4973-abb3-903de845a908 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.296720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55d771-23d3-4973-abb3-903de845a908 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.310941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55d771-23d3-4973-abb3-903de845a908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ae55d771-23d3-4973-abb3-903de845a908 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:05.316488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:05.317516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:05.318436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 226/440] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:05.319849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471c199-8e83-4359-8802-2d4c148cba0b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.324220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471c199-8e83-4359-8802-2d4c148cba0b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.324505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471c199-8e83-4359-8802-2d4c148cba0b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.325828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 227/441] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:05.326549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471c199-8e83-4359-8802-2d4c148cba0b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.339871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a184a0-a706-439e-90ec-c2f93853d7dd None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.379831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fbe87-7c94-45ef-9f60-355a07cc8e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.380387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbe87-7c94-45ef-9f60-355a07cc8e0b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.380667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbe87-7c94-45ef-9f60-355a07cc8e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.380936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbe87-7c94-45ef-9f60-355a07cc8e0b None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.382920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fbe87-7c94-45ef-9f60-355a07cc8e0b None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.384412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fbe87-7c94-45ef-9f60-355a07cc8e0b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.386166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 215/442] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:05.387160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a8b6-f314-4f5f-adb4-c38b1e9ee546 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.387443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a8b6-f314-4f5f-adb4-c38b1e9ee546 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.387712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84a8b6-f314-4f5f-adb4-c38b1e9ee546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.408583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-eb84a8b6-f314-4f5f-adb4-c38b1e9ee546 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:05.409773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-eb84a8b6-f314-4f5f-adb4-c38b1e9ee546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:05.446116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.451154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471c199-8e83-4359-8802-2d4c148cba0b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.483301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 216/443] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:05.484134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a8b6-f314-4f5f-adb4-c38b1e9ee546 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.485357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.485964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.486241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.486504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.487185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:05.487480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.488822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.499567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 228/444] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:05.505028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.552541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.592820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84a8b6-f314-4f5f-adb4-c38b1e9ee546 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.612307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a3528-a52f-48fe-969b-17e47f8da916 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.627547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.628145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.628422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.628691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.629414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] RBAC: Adding query filter params (group.id=ffeedce6d78742afac2b704ef77ab9f2, role.id=aa4ffe964b624a329cecff46b0a2c131, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:05.629712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.631201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.646933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 217/445] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/role_assignments?group.id=ffeedce6d78742afac2b704ef77ab9f2&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=aa4ffe964b624a329cecff46b0a2c131 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:05.651927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa1b39-eb0e-4dee-8ce9-81f8401a153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.652545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa1b39-eb0e-4dee-8ce9-81f8401a153c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.652822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa1b39-eb0e-4dee-8ce9-81f8401a153c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.653090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa1b39-eb0e-4dee-8ce9-81f8401a153c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.685431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.730407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa1b39-eb0e-4dee-8ce9-81f8401a153c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.731899 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa1b39-eb0e-4dee-8ce9-81f8401a153c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.753328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.843860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 229/446] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:46:05.850744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa1b39-eb0e-4dee-8ce9-81f8401a153c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.865668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1200d4-602a-4ca4-9ae4-4fc60c11d859 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.894035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.904506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73d7ac-63da-4e78-967d-96020a364ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.905131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73d7ac-63da-4e78-967d-96020a364ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.905403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73d7ac-63da-4e78-967d-96020a364ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.905666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73d7ac-63da-4e78-967d-96020a364ad3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.918953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73d7ac-63da-4e78-967d-96020a364ad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:05.919451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-cc73d7ac-63da-4e78-967d-96020a364ad3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:05.924539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:05.925435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:05.926201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:05.926857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 218/447] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:05.932845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73d7ac-63da-4e78-967d-96020a364ad3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.971470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa1b39-eb0e-4dee-8ce9-81f8401a153c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.011860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca141c30-4c28-4e36-971c-cffb831f9f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.012721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca141c30-4c28-4e36-971c-cffb831f9f70 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.013173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca141c30-4c28-4e36-971c-cffb831f9f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.013451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca141c30-4c28-4e36-971c-cffb831f9f70 None admin] PATH_INFO: `/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.014588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca141c30-4c28-4e36-971c-cffb831f9f70 None admin] RBAC: Authorizing `identity:get_project(project_id=fcc729b56f9c4ffb9e49f733ba16d591)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.016278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca141c30-4c28-4e36-971c-cffb831f9f70 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.018228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 230/448] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/projects/fcc729b56f9c4ffb9e49f733ba16d591 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:06.024127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca141c30-4c28-4e36-971c-cffb831f9f70 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.046831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73d7ac-63da-4e78-967d-96020a364ad3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.079991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.080591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.080873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.081145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.081804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:06.082106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.083502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.091926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 219/449] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:06.097657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.124327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca141c30-4c28-4e36-971c-cffb831f9f70 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.145918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.151392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.159477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9aee3d-5b9c-4c2d-bbd2-36375e51639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.160101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9aee3d-5b9c-4c2d-bbd2-36375e51639d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.160381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9aee3d-5b9c-4c2d-bbd2-36375e51639d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.160653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9aee3d-5b9c-4c2d-bbd2-36375e51639d None admin] PATH_INFO: `/v3/groups/ffeedce6d78742afac2b704ef77ab9f2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.161742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9aee3d-5b9c-4c2d-bbd2-36375e51639d None admin] RBAC: Authorizing `identity:get_group(group_id=ffeedce6d78742afac2b704ef77ab9f2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.163197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9aee3d-5b9c-4c2d-bbd2-36375e51639d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.164758 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 231/450] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/groups/ffeedce6d78742afac2b704ef77ab9f2 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:06.170417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9aee3d-5b9c-4c2d-bbd2-36375e51639d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.196927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a50ed-39b2-48b3-8bff-0f1795bb315a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.229476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452700f-54de-44dd-bb19-82643299d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.230067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452700f-54de-44dd-bb19-82643299d266 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.230344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452700f-54de-44dd-bb19-82643299d266 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.230658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452700f-54de-44dd-bb19-82643299d266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.270046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9aee3d-5b9c-4c2d-bbd2-36375e51639d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.297886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452700f-54de-44dd-bb19-82643299d266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.299355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452700f-54de-44dd-bb19-82643299d266 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.303320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.303911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.304182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.304445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.305155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] RBAC: Adding query filter params (group.id=ffeedce6d78742afac2b704ef77ab9f2, role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:06.305456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.306829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.312517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 232/451] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/role_assignments?group.id=ffeedce6d78742afac2b704ef77ab9f2&scope.project.id=fcc729b56f9c4ffb9e49f733ba16d591&role.id=2e40648646094078acac22f8eb94dd98 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:06.349974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.407004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 220/452] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:46:06.413592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.476353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.518101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74111b63-dfb7-409a-aca7-f0282bdd7eac None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.551638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bdb30-4e1c-4dcd-8724-170e89d6cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.552271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bdb30-4e1c-4dcd-8724-170e89d6cb25 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.552550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bdb30-4e1c-4dcd-8724-170e89d6cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.552827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bdb30-4e1c-4dcd-8724-170e89d6cb25 None admin] PATH_INFO: `/v3/projects/992fcf5a85d6497db94cb36327aec1f0` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.554483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bdb30-4e1c-4dcd-8724-170e89d6cb25 None admin] RBAC: Authorizing `identity:get_project(project_id=992fcf5a85d6497db94cb36327aec1f0)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.555934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bdb30-4e1c-4dcd-8724-170e89d6cb25 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.557561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 233/453] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/projects/992fcf5a85d6497db94cb36327aec1f0 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:06.563165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452700f-54de-44dd-bb19-82643299d266 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.664824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452700f-54de-44dd-bb19-82643299d266 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.698752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba780b-f11f-4de0-90a8-74b8fc204a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.699403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba780b-f11f-4de0-90a8-74b8fc204a74 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.699684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba780b-f11f-4de0-90a8-74b8fc204a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.699955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba780b-f11f-4de0-90a8-74b8fc204a74 None admin] PATH_INFO: `/v3/groups/b9eb1033fcf6474eb12163421df03e40` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.701052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba780b-f11f-4de0-90a8-74b8fc204a74 None admin] RBAC: Authorizing `identity:get_group(group_id=b9eb1033fcf6474eb12163421df03e40)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.702489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba780b-f11f-4de0-90a8-74b8fc204a74 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.704056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 221/454] 10.222.0.52 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/groups/b9eb1033fcf6474eb12163421df03e40 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:06.709873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bdb30-4e1c-4dcd-8724-170e89d6cb25 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.805857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bdb30-4e1c-4dcd-8724-170e89d6cb25 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.831115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.839317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594aeb4-521e-4117-b24f-c01dd5d1f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.839910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594aeb4-521e-4117-b24f-c01dd5d1f15b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.840192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594aeb4-521e-4117-b24f-c01dd5d1f15b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.840467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594aeb4-521e-4117-b24f-c01dd5d1f15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.841185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594aeb4-521e-4117-b24f-c01dd5d1f15b None admin] RBAC: Adding query filter params (group.id=b9eb1033fcf6474eb12163421df03e40, role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=992fcf5a85d6497db94cb36327aec1f0) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:06.841491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594aeb4-521e-4117-b24f-c01dd5d1f15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.843191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594aeb4-521e-4117-b24f-c01dd5d1f15b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.848726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 234/455] 10.222.0.52 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/role_assignments?group.id=b9eb1033fcf6474eb12163421df03e40&scope.project.id=992fcf5a85d6497db94cb36327aec1f0&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:09.444531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a38c5-82d4-4065-858b-0a89b6f186cd None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.445511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a38c5-82d4-4065-858b-0a89b6f186cd None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.445977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a38c5-82d4-4065-858b-0a89b6f186cd None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.453015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1f54-a71a-46c7-bab8-2f085d017e52 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.454013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1f54-a71a-46c7-bab8-2f085d017e52 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.454478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1f54-a71a-46c7-bab8-2f085d017e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.457471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 222/456] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:09.484087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-797d1f54-a71a-46c7-bab8-2f085d017e52 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:09.485706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-797d1f54-a71a-46c7-bab8-2f085d017e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:09.580572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 235/457] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:09.597443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768296-dd8c-4a38-8a10-69374d460b9a None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.598005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768296-dd8c-4a38-8a10-69374d460b9a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.598470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768296-dd8c-4a38-8a10-69374d460b9a None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.599998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 223/458] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:09.605193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde293-df77-45e4-bac6-82518ab3b680 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.605702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde293-df77-45e4-bac6-82518ab3b680 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.606163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fde293-df77-45e4-bac6-82518ab3b680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.627031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-34fde293-df77-45e4-bac6-82518ab3b680 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:09.628238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-34fde293-df77-45e4-bac6-82518ab3b680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:09.709229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 236/459] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:09.984568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5768296-dd8c-4a38-8a10-69374d460b9a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.997424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fde293-df77-45e4-bac6-82518ab3b680 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.014028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.086146 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.139470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fde293-df77-45e4-bac6-82518ab3b680 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.148604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5768296-dd8c-4a38-8a10-69374d460b9a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.183728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.185241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.185839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.186375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.192464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d0fd3-5d78-43c8-a6c9-8736e7b4a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.193164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d0fd3-5d78-43c8-a6c9-8736e7b4a68a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.193510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d0fd3-5d78-43c8-a6c9-8736e7b4a68a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.194378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d0fd3-5d78-43c8-a6c9-8736e7b4a68a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.201201 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.203956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.205737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d0fd3-5d78-43c8-a6c9-8736e7b4a68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.207461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d0fd3-5d78-43c8-a6c9-8736e7b4a68a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-226d0fd3-5d78-43c8-a6c9-8736e7b4a68a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:10.221985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:10.222921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:10.223765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:10.223765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:10.223765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:10.224918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:10.225779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:10.226636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:10.226636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:10.226636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:10.227376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 224/460] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:10.228641 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0595d1c-fe86-45ff-a0b5-878a5923f7d7 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.228971 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 237/461] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:10.229069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0595d1c-fe86-45ff-a0b5-878a5923f7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.229313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0595d1c-fe86-45ff-a0b5-878a5923f7d7 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.230613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 225/462] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:10.235855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.237335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0595d1c-fe86-45ff-a0b5-878a5923f7d7 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.312340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.363571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0595d1c-fe86-45ff-a0b5-878a5923f7d7 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.374565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e93f-1e5c-4040-9401-515db0e32aa1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.377768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.405594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a5ba7-89e4-4bdc-9d3a-54fbf4e2b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.406307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a5ba7-89e4-4bdc-9d3a-54fbf4e2b5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.406651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a5ba7-89e4-4bdc-9d3a-54fbf4e2b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.407031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a5ba7-89e4-4bdc-9d3a-54fbf4e2b5a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.408193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a5ba7-89e4-4bdc-9d3a-54fbf4e2b5a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:10.408193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a5ba7-89e4-4bdc-9d3a-54fbf4e2b5a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.414914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a5ba7-89e4-4bdc-9d3a-54fbf4e2b5a0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.427464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 226/463] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:10.428824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d09e16-3385-40ef-b64a-3cd8ed951a52 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.429445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.430059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.430338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.430608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.431145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d09e16-3385-40ef-b64a-3cd8ed951a52 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.431323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:10.431591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.431975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d09e16-3385-40ef-b64a-3cd8ed951a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.432962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.453350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 238/464] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:10.453350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.456807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c1d09e16-3385-40ef-b64a-3cd8ed951a52 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:10.458457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c1d09e16-3385-40ef-b64a-3cd8ed951a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:10.529008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 227/465] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:10.530305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d09e16-3385-40ef-b64a-3cd8ed951a52 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.569524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662bceb-2ac3-40fb-b84f-f68637ae9a07 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.608220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.613281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd3559-9ef9-46e0-bd81-e302e01fbf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.613964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3559-9ef9-46e0-bd81-e302e01fbf20 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.614312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3559-9ef9-46e0-bd81-e302e01fbf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.614670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd3559-9ef9-46e0-bd81-e302e01fbf20 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.615393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd3559-9ef9-46e0-bd81-e302e01fbf20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.616906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd3559-9ef9-46e0-bd81-e302e01fbf20 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.641930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 239/466] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:10 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:10.649460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c323b0-ce4d-47b5-ac53-12bddbf1db07 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.650104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c323b0-ce4d-47b5-ac53-12bddbf1db07 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.650446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c323b0-ce4d-47b5-ac53-12bddbf1db07 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.652156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 240/467] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:10.658639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d09e16-3385-40ef-b64a-3cd8ed951a52 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.658816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9bea6-fc2f-4de4-a05f-ad8e028efc0d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.658998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9bea6-fc2f-4de4-a05f-ad8e028efc0d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.659459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9bea6-fc2f-4de4-a05f-ad8e028efc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.692172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-53b9bea6-fc2f-4de4-a05f-ad8e028efc0d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:10.693540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-53b9bea6-fc2f-4de4-a05f-ad8e028efc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:10.701840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040964-3524-4073-8d33-acc4602d369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.702868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040964-3524-4073-8d33-acc4602d369c None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.703396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040964-3524-4073-8d33-acc4602d369c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.703875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040964-3524-4073-8d33-acc4602d369c None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.704779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040964-3524-4073-8d33-acc4602d369c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.706591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040964-3524-4073-8d33-acc4602d369c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.726670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.741595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 228/468] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:10 2022] POST /identity/v3/users => generated 246 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:10.827306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040964-3524-4073-8d33-acc4602d369c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.857623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 241/469] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:10.859184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fa285-657c-4256-a2a9-adc4483e682d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.859493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fa285-657c-4256-a2a9-adc4483e682d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.859794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fa285-657c-4256-a2a9-adc4483e682d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.861522 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 242/470] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:10.867407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7db934-bb9f-4810-bf75-d8796045bd4d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.867786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7db934-bb9f-4810-bf75-d8796045bd4d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.868454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7db934-bb9f-4810-bf75-d8796045bd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.910963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fe7db934-bb9f-4810-bf75-d8796045bd4d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:10.912489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fe7db934-bb9f-4810-bf75-d8796045bd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:10.990482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040964-3524-4073-8d33-acc4602d369c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.003764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 243/471] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:11.011240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.035604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.036235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.036515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.036787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.050007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.051539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:11.067337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:11.068238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:11.069100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:11.069100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:11.069100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:11.069100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 229/472] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:11.075021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7db934-bb9f-4810-bf75-d8796045bd4d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.169073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.177604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.190040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7db934-bb9f-4810-bf75-d8796045bd4d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.227322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.227936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.228214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.228492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.229140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:11.229444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.230809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.243320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 244/473] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:11.256198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.323521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacecd11-3db4-40d5-83df-a5fd67b1275a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.360177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.360799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.361172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.361453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.365466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8155c-c009-479b-b1ec-dcad4f4b423f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.373528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.375035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:11.394873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:11.395800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:11.396617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:11.396617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:11.396617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:11.396617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 230/474] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:11.397853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.407476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ee653-e4e4-4e47-af06-cc8eaf4e53e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.408187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ee653-e4e4-4e47-af06-cc8eaf4e53e7 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.408547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ee653-e4e4-4e47-af06-cc8eaf4e53e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.408904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ee653-e4e4-4e47-af06-cc8eaf4e53e7 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.409628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ee653-e4e4-4e47-af06-cc8eaf4e53e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.411230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ee653-e4e4-4e47-af06-cc8eaf4e53e7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.424001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.427258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.441905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 245/475] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:11 2022] POST /identity/v3/users => generated 248 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:11.443068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ee653-e4e4-4e47-af06-cc8eaf4e53e7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.527691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bf068-010a-4e04-b53a-cbd75b01e149 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.560643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ee653-e4e4-4e47-af06-cc8eaf4e53e7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.599630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.602915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.602915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.602915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.602915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:11.602915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.604167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.614213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.615323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.616403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.616978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.617842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.619470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.624258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 246/476] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:11.625751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.628052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.sql.core [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 11:46:11.628052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 10 11:46:11.628052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [parameters: {'id': '17d7335aab4545a29db690d2d335d700', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 10 11:46:11.628052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81457) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result.read() Oct 10 11:46:11.645749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return ref.to_dict() Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:46:11.646689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application yield resource Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise self.value Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application session.commit() Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 11:46:11.647668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.session.flush() Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application compat.raise_( Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 11:46:11.648533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:46:11.650260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result.read() Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:46:11.651559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [parameters: {'id': '17d7335aab4545a29db690d2d335d700', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:11.653591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 10 11:46:11.656027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:11.664548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 231/477] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Oct 10 11:46:11 2022] POST /identity/v3/roles => generated 144 bytes in 267 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:11.670487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.691823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.753246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f89d34-d4df-4a15-83f2-b56dd5ca9d62 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.797122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.801886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68cfd-2b3c-4fdc-adae-388ee59aeb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.802658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68cfd-2b3c-4fdc-adae-388ee59aeb3e None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.802975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68cfd-2b3c-4fdc-adae-388ee59aeb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.803811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68cfd-2b3c-4fdc-adae-388ee59aeb3e None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.804390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68cfd-2b3c-4fdc-adae-388ee59aeb3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.806164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68cfd-2b3c-4fdc-adae-388ee59aeb3e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.809300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09161918-6b8c-4321-9b59-a4019a59246f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.835050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 247/478] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:11 2022] POST /identity/v3/users => generated 249 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:46:11.861903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5906e4-12f4-45ce-9145-f32f46f15324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.862603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5906e4-12f4-45ce-9145-f32f46f15324 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.862885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5906e4-12f4-45ce-9145-f32f46f15324 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.863226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5906e4-12f4-45ce-9145-f32f46f15324 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.876324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5906e4-12f4-45ce-9145-f32f46f15324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-9e5906e4-12f4-45ce-9145-f32f46f15324 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:11.881713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:11.882649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:11.883724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 232/479] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:11.888677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68cfd-2b3c-4fdc-adae-388ee59aeb3e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:12.012114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68cfd-2b3c-4fdc-adae-388ee59aeb3e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:12.030707 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:12.055335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4f26f-d70b-43d0-930b-a597ed9f468b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:12.055956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4f26f-d70b-43d0-930b-a597ed9f468b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:12.056234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4f26f-d70b-43d0-930b-a597ed9f468b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:12.056508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4f26f-d70b-43d0-930b-a597ed9f468b None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:12.057174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4f26f-d70b-43d0-930b-a597ed9f468b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:12.057474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4f26f-d70b-43d0-930b-a597ed9f468b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:12.058824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4f26f-d70b-43d0-930b-a597ed9f468b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:12.068697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 248/480] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:13.551733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5a743-7d85-4f9e-8645-a6ab6115fea3 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.552241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5a743-7d85-4f9e-8645-a6ab6115fea3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.552802 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5a743-7d85-4f9e-8645-a6ab6115fea3 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.554571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 233/481] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:13.560017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a5cbd-577c-4fc9-b47c-616c4d7cfffd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.560416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a5cbd-577c-4fc9-b47c-616c4d7cfffd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.560559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a5cbd-577c-4fc9-b47c-616c4d7cfffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.585704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-175a5cbd-577c-4fc9-b47c-616c4d7cfffd None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:13.586882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-175a5cbd-577c-4fc9-b47c-616c4d7cfffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:13.657135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d31aa-7800-4701-acfd-52fb05e57799 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.657336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d31aa-7800-4701-acfd-52fb05e57799 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.657609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d31aa-7800-4701-acfd-52fb05e57799 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.658944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 234/482] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:13.664293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d86d5-0e24-440a-839a-441d8b4bcb68 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.664575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d86d5-0e24-440a-839a-441d8b4bcb68 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.664847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d86d5-0e24-440a-839a-441d8b4bcb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.668048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 249/483] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:13.692934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6c3d86d5-0e24-440a-839a-441d8b4bcb68 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:13.694758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6c3d86d5-0e24-440a-839a-441d8b4bcb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:13.773761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 235/484] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:14.042865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a5cbd-577c-4fc9-b47c-616c4d7cfffd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.070591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d86d5-0e24-440a-839a-441d8b4bcb68 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.173262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a5cbd-577c-4fc9-b47c-616c4d7cfffd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.176026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.219494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf996d-e6e2-4cea-b28b-c855d16a8b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.220508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf996d-e6e2-4cea-b28b-c855d16a8b96 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.220993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf996d-e6e2-4cea-b28b-c855d16a8b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.221496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf996d-e6e2-4cea-b28b-c855d16a8b96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.235255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.241786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf996d-e6e2-4cea-b28b-c855d16a8b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-bebf996d-e6e2-4cea-b28b-c855d16a8b96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:14.248009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:14.248866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:14.249637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:14.252269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 250/485] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:14.258524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf996d-e6e2-4cea-b28b-c855d16a8b96 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.293765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d86d5-0e24-440a-839a-441d8b4bcb68 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.378450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbcd3f-1749-4d40-8b38-e2ee9d44e0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.379476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcd3f-1749-4d40-8b38-e2ee9d44e0ef None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.379954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcd3f-1749-4d40-8b38-e2ee9d44e0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.380416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcd3f-1749-4d40-8b38-e2ee9d44e0ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.401435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbcd3f-1749-4d40-8b38-e2ee9d44e0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b1fbcd3f-1749-4d40-8b38-e2ee9d44e0ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:14.416132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:14.417058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:14.417820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:14.420492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 236/486] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:14.440678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbcd3f-1749-4d40-8b38-e2ee9d44e0ef None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.476591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf996d-e6e2-4cea-b28b-c855d16a8b96 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.516779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ecebe-d3b8-4223-8aa0-1108f7702409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.517765 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ecebe-d3b8-4223-8aa0-1108f7702409 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.518188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ecebe-d3b8-4223-8aa0-1108f7702409 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.518625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ecebe-d3b8-4223-8aa0-1108f7702409 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.519455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ecebe-d3b8-4223-8aa0-1108f7702409 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:14.519894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ecebe-d3b8-4223-8aa0-1108f7702409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.521391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ecebe-d3b8-4223-8aa0-1108f7702409 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.530916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 251/487] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:14.532376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783646-d80a-4cb0-99b0-21b31672386b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.532820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783646-d80a-4cb0-99b0-21b31672386b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.533092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783646-d80a-4cb0-99b0-21b31672386b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.534927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 252/488] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:14.536367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c783646-d80a-4cb0-99b0-21b31672386b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.564743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbcd3f-1749-4d40-8b38-e2ee9d44e0ef None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.569343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.610860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9555a-0d10-4185-88a3-e0dbd4cc4656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.611513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9555a-0d10-4185-88a3-e0dbd4cc4656 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.612065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9555a-0d10-4185-88a3-e0dbd4cc4656 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.612343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9555a-0d10-4185-88a3-e0dbd4cc4656 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.613009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9555a-0d10-4185-88a3-e0dbd4cc4656 None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:14.613316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9555a-0d10-4185-88a3-e0dbd4cc4656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.614768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9555a-0d10-4185-88a3-e0dbd4cc4656 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.617146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.627738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 237/489] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:14.632874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eef3d-8eb2-48f0-a490-c9fe1eb8738e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.632874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eef3d-8eb2-48f0-a490-c9fe1eb8738e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.632874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eef3d-8eb2-48f0-a490-c9fe1eb8738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.656233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-973eef3d-8eb2-48f0-a490-c9fe1eb8738e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:14.658530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-973eef3d-8eb2-48f0-a490-c9fe1eb8738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:14.670953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c783646-d80a-4cb0-99b0-21b31672386b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.710831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a35455-0285-4f36-90f9-4a2771015845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.712200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a35455-0285-4f36-90f9-4a2771015845 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.712877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a35455-0285-4f36-90f9-4a2771015845 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.713527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a35455-0285-4f36-90f9-4a2771015845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.726976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a35455-0285-4f36-90f9-4a2771015845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.728643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a35455-0285-4f36-90f9-4a2771015845 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-81a35455-0285-4f36-90f9-4a2771015845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:14.745871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:14.747023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:14.747930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:14.747930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:14.747930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:14.748087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 238/490] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:14.755246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eef3d-8eb2-48f0-a490-c9fe1eb8738e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.755387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 253/491] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:14.757623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e94d3-f150-4cfb-906a-5b1eb2c42ed9 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.757912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e94d3-f150-4cfb-906a-5b1eb2c42ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.758187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e94d3-f150-4cfb-906a-5b1eb2c42ed9 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.759791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 254/492] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:14.761501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e94d3-f150-4cfb-906a-5b1eb2c42ed9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.850245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.861168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eef3d-8eb2-48f0-a490-c9fe1eb8738e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.875440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e94d3-f150-4cfb-906a-5b1eb2c42ed9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.888146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.898264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf8009-5ab4-44d6-823d-db74d39f2035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.898901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf8009-5ab4-44d6-823d-db74d39f2035 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.899210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf8009-5ab4-44d6-823d-db74d39f2035 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.899484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf8009-5ab4-44d6-823d-db74d39f2035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.909788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf8009-5ab4-44d6-823d-db74d39f2035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.910805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5636a26-d11d-498d-8320-bc2ab661a736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.911252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf8009-5ab4-44d6-823d-db74d39f2035 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.911439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5636a26-d11d-498d-8320-bc2ab661a736 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.911748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5636a26-d11d-498d-8320-bc2ab661a736 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.912025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5636a26-d11d-498d-8320-bc2ab661a736 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.912692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5636a26-d11d-498d-8320-bc2ab661a736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:14.912997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5636a26-d11d-498d-8320-bc2ab661a736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.914343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5636a26-d11d-498d-8320-bc2ab661a736 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b7cf8009-5ab4-44d6-823d-db74d39f2035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:14.925778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:14.926623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:14.927557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:14.927557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:14.927557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:14.931906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 239/494] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:14.932036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 255/494] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:14.933200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbdf8-8d55-4e07-a2f5-19b175f1f42c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.933298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d7f5-a657-4431-9dd7-0f343f0be05f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.933490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbdf8-8d55-4e07-a2f5-19b175f1f42c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.933580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d7f5-a657-4431-9dd7-0f343f0be05f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.933766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dbdf8-8d55-4e07-a2f5-19b175f1f42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.933856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d7f5-a657-4431-9dd7-0f343f0be05f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.938992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 256/495] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:14.938992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5d7f5-a657-4431-9dd7-0f343f0be05f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.954592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-786dbdf8-8d55-4e07-a2f5-19b175f1f42c None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:14.955799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-786dbdf8-8d55-4e07-a2f5-19b175f1f42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:15.024070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 240/496] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:15.030409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dbdf8-8d55-4e07-a2f5-19b175f1f42c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.046624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5d7f5-a657-4431-9dd7-0f343f0be05f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.084701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7d2ae-e4b0-490f-bd19-f88ce601b887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.085598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7d2ae-e4b0-490f-bd19-f88ce601b887 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.085881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7d2ae-e4b0-490f-bd19-f88ce601b887 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.086154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7d2ae-e4b0-490f-bd19-f88ce601b887 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.086792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7d2ae-e4b0-490f-bd19-f88ce601b887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:15.087122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7d2ae-e4b0-490f-bd19-f88ce601b887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:15.088985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7d2ae-e4b0-490f-bd19-f88ce601b887 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:15.095690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:15.103669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 257/497] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:15.104782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90d97a-4235-4b6d-94bf-7cbaf09b54cb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.105068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90d97a-4235-4b6d-94bf-7cbaf09b54cb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.105340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90d97a-4235-4b6d-94bf-7cbaf09b54cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.123209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3c90d97a-4235-4b6d-94bf-7cbaf09b54cb None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:15.124442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3c90d97a-4235-4b6d-94bf-7cbaf09b54cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:15.142183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dbdf8-8d55-4e07-a2f5-19b175f1f42c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.151385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:15.177014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.177759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.178701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.179100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.186117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:15.187661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:15.206915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 258/498] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:15.206915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90d97a-4235-4b6d-94bf-7cbaf09b54cb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:15.207141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:15.208151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 241/499] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:15.209228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.319515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a8887-d24a-4146-92ed-ea53e226932f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.330551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90d97a-4235-4b6d-94bf-7cbaf09b54cb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.353013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.353638 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.353974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.354197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.359645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:15.361743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:15.363236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:15.369423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58babc4c-9df2-4f84-be37-80270343308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.370115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58babc4c-9df2-4f84-be37-80270343308c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.370455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58babc4c-9df2-4f84-be37-80270343308c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.370789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58babc4c-9df2-4f84-be37-80270343308c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:15.375003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:15.376070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:15.377138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:15.380797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 242/500] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:15.381565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.384620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58babc4c-9df2-4f84-be37-80270343308c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-58babc4c-9df2-4f84-be37-80270343308c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:15.389532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:15.390976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 259/501] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:15.391914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58babc4c-9df2-4f84-be37-80270343308c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.421623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:15.497975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c7172-becc-4471-ace0-0a1f66aecc06 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.507239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58babc4c-9df2-4f84-be37-80270343308c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.531303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.531904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.532181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.532451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.533198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:15.533504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:15.534948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:15.541606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493eb82-4980-474d-b0a3-b4a909af75be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.542225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493eb82-4980-474d-b0a3-b4a909af75be None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.542502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493eb82-4980-474d-b0a3-b4a909af75be None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.542769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493eb82-4980-474d-b0a3-b4a909af75be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.553407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493eb82-4980-474d-b0a3-b4a909af75be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1493eb82-4980-474d-b0a3-b4a909af75be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:15.558124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:15.559254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 260/502] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:15.560043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493eb82-4980-474d-b0a3-b4a909af75be None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.572616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 243/503] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:15.573437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.610980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:15.655462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:15.662059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493eb82-4980-474d-b0a3-b4a909af75be None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.671487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874626d-0004-43e1-8c21-f54e79ed7fc3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.697181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.697777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.698053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.698323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.699077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:15.699379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:15.700799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:15.704091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.704686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.704968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.705241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.705897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:15.706202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:15.707595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:15.715806 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 244/504] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:15.716603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.738381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 261/505] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:15.739504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.817938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f687a2-98dd-440a-9b8e-a83b3875c91d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.837488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87ebb2-360f-4c2a-8154-b1b3d729230f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.844561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:15.851602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.852229 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.852509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.852785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.863319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:15.864709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:15.872501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:15.872611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:15.873107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:15.873389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:15.873676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:15.874318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:15.874617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:15.876007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:15.879070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:15.879961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:15.880786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:15.880786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:15.880786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:15.880786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 245/506] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:15.883746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.889298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 262/507] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:15.889298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:15.980829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b29a-b955-4414-be0d-e8bfd36246b6 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:15.988261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e741b83-1e69-47c5-8945-d1eb5b1ec109 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.012967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0be6f5-df3c-4cb2-90fa-3eceafee9778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.013560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0be6f5-df3c-4cb2-90fa-3eceafee9778 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.013839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0be6f5-df3c-4cb2-90fa-3eceafee9778 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.014110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0be6f5-df3c-4cb2-90fa-3eceafee9778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.020997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.021615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.022459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.022771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.033078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.034516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:16.048564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:16.049622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:16.050472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:16.050472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:16.050472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:16.050472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 263/508] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:16.050808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.073807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.081453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.084122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0be6f5-df3c-4cb2-90fa-3eceafee9778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.086552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0be6f5-df3c-4cb2-90fa-3eceafee9778 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.152497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb67a7-fa22-4ef2-8e18-d8952def5d7e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.181399 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 246/509] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:16.182360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0be6f5-df3c-4cb2-90fa-3eceafee9778 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.185024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2920711-fc4f-43cf-af47-5e4cd6d1369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.185636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2920711-fc4f-43cf-af47-5e4cd6d1369b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.186138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2920711-fc4f-43cf-af47-5e4cd6d1369b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.186411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2920711-fc4f-43cf-af47-5e4cd6d1369b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.252539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2920711-fc4f-43cf-af47-5e4cd6d1369b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.253989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2920711-fc4f-43cf-af47-5e4cd6d1369b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.274639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.279949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.283078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0be6f5-df3c-4cb2-90fa-3eceafee9778 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.316115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.316713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.316994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.317259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.317882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:16.318175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.320179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.330773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 247/510] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:16.335368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.360570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 264/511] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:16.361395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2920711-fc4f-43cf-af47-5e4cd6d1369b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.435715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468653c-7b48-4a43-ba0d-d92953419197 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.458651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2920711-fc4f-43cf-af47-5e4cd6d1369b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.469672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.470289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.470563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.470830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.477955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.480769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.482223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.492603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.493199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.493478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.493748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.494378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:16.494674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.496054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:16.497353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:16.498325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:16.504039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:16.504039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:16.504039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:16.504039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 248/512] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:16.504039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.507975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 265/513] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:16.508756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.518192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.600888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6d879-60fa-4d52-bfcb-2ac7b63d4026 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.604958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cff86-919c-4acf-ab28-6546522f1449 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.633761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.634372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.634648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.634937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.637670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728fcd2-c0dc-48bb-97f0-cc93342d2af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.638259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728fcd2-c0dc-48bb-97f0-cc93342d2af3 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.638537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728fcd2-c0dc-48bb-97f0-cc93342d2af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.638804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728fcd2-c0dc-48bb-97f0-cc93342d2af3 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.639469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728fcd2-c0dc-48bb-97f0-cc93342d2af3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.640885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728fcd2-c0dc-48bb-97f0-cc93342d2af3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.645007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.646411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:16.660411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:16.661295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:16.662141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:16.662141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:16.662141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:16.662838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 266/514] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:16 2022] POST /identity/v3/users => generated 247 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:16.662935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 249/515] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:16.663004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728fcd2-c0dc-48bb-97f0-cc93342d2af3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.663069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.711932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.741405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.785467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0c8a8-2f9c-4ba3-9bc9-e3547aa171e4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.794004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728fcd2-c0dc-48bb-97f0-cc93342d2af3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.823082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.823833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.824121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.824383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.825030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:16.825330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.826758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.836612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:16.837248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.837528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.837799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.840055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 250/516] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:16.840970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.848506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:16.849984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:16.867997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:16.868902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:16.869725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:16.869725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:16.869725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:16.869725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 267/517] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:16.869946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:16.962653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a99e2d-4a07-4f57-befc-30fa311db89b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.963243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:16.975434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4230a2-2e52-4b1b-8ba8-1f6cf5173b98 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:16.992136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.003050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.003680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.003950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.004215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.011764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.013132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.013249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.013735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.014016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.014370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.015191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:17.015496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.016850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:17.024756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:17.025635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:17.026518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 251/518] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:17.028074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.037807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 268/519] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:17.037807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.138199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121168f-0d5e-4c18-9a1e-d8c582b95131 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.144120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b24b9b-c8f2-466c-8223-822500accc55 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.175145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.175774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.176056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.176367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.181363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.181966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.182249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.182517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.183211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:17.183492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.184106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.184842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.185537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.201886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 269/520] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:17.201886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:17.202114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:17.202960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 252/521] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:17.204321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.215066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.250803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.307317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d441b-5198-48ca-b6b2-6b94ffe10041 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.312022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703c792-da7b-41b8-bb02-1988298d9f1c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.342549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.343206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.343485 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.343752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.346691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.347324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.347602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.347873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.348591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:17.348897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.350386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.350488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.351984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:17.363875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:17.364730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:17.366110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:17.366110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:17.366110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:17.366110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:17.366110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:17.366110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:17.366110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:17.366110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:17.370283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 253/522] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:17.375414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.375539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 270/523] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:17.375539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.444096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.487216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c00ca-fd06-48b4-8ccd-da4a48d8cc30 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.488833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ebb00-6e3b-4e36-b322-9bd27f9a9ac0 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.491718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.524804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.525033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.525410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.525511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.525687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.525778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.525955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.526045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.526704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:17.527035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.528445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.532936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.534375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.546235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 271/524] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:17.547051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:17.547868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:17.548677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:17.548677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:17.548677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:17.548677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:17.548677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:17.548677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:17.548677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:17.548677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:17.550224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.551570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 254/525] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:17.552547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.656661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947d88c-2187-4e35-9661-931cc8604e68 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.659283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718f7eb-8d81-4fa9-a140-693d9b4bbaca None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.684410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.695017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a33a-26fc-43c8-8bef-47184417ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.695623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a33a-26fc-43c8-8bef-47184417ef3a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.695895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a33a-26fc-43c8-8bef-47184417ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.696218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a33a-26fc-43c8-8bef-47184417ef3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.697268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.697870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.698153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.698426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.699159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:17.699464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.700893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.724228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 272/526] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:17.724228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.733104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.775963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596a33a-26fc-43c8-8bef-47184417ef3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.777333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596a33a-26fc-43c8-8bef-47184417ef3a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.829444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfac07c-0554-49d4-9a56-bbc4f128d0a4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.865984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b7b86-b44b-4a22-8f5e-8c43e97e2a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.866644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b7b86-b44b-4a22-8f5e-8c43e97e2a2d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.866944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b7b86-b44b-4a22-8f5e-8c43e97e2a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.867209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b7b86-b44b-4a22-8f5e-8c43e97e2a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.882416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 255/527] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:17.882416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a33a-26fc-43c8-8bef-47184417ef3a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.931284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.943434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.948186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b7b86-b44b-4a22-8f5e-8c43e97e2a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.949658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b7b86-b44b-4a22-8f5e-8c43e97e2a2d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.983002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a33a-26fc-43c8-8bef-47184417ef3a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.018247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.018850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.019159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.019428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.020121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:18.020425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.022376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.045230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 256/528] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:18.045230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.066795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 273/529] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:18.067793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b7b86-b44b-4a22-8f5e-8c43e97e2a2d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.147799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6fe29-495a-4ff9-92fd-348edd366764 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.152054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.168459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b7b86-b44b-4a22-8f5e-8c43e97e2a2d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.172483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.184205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.184802 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.185084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.185354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.186061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=152edb19fd7b434485b0e1ebc5078335) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:18.186364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.187812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.193516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 257/530] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/role_assignments?user.id=152edb19fd7b434485b0e1ebc5078335&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:18.194630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.215859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.217215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.218253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.218743 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.232827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.234812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:18.255662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:18.257175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:18.258020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:18.258020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:18.258020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:18.265346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 274/531] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:18.265346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.332162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75567336-b1b4-4dde-b0bd-1a4d3edb07a9 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.373770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.374704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.375119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.375493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.376286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=c6774fdd8b2e46b6a7d00919568f9fd1) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:18.376613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.377949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3de49d-a498-43f6-be59-081fdba9ff09 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.378251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.384924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 258/532] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/role_assignments?user.id=c6774fdd8b2e46b6a7d00919568f9fd1&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:18.385704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.423260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.424084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.424562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.424934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.433094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.438663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.440470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.456976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:18.458764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:18.460252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:18.461093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:18.461093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:18.461093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:18.461093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 275/533] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:18.467854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.556105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08e08b-76b6-4140-b697-6cdb2ed0e417 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.609282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.609923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.610205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.610474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.611148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:18.611548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.612925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.631590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 259/534] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:18.633397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.634363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af76f4-7ad1-4910-88da-370e4d7f7372 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.670586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.671223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.671487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.671738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.672372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:18.672667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.674025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.688819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 276/535] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:18.695047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.786303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd5026-89f0-482a-b14c-fb604ad8e68f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.801730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.812519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.838436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.839562 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.839842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.840110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.848979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.850519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:18.864457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:18.867077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:18.869118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 260/536] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:18.874927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.917199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca074e-458c-41c5-aeba-d4dae35337f4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.984351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27946a1c-eb5b-4b5d-bf13-4b6e4c72fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.986949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27946a1c-eb5b-4b5d-bf13-4b6e4c72fe6b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.987721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27946a1c-eb5b-4b5d-bf13-4b6e4c72fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.989041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27946a1c-eb5b-4b5d-bf13-4b6e4c72fe6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.998747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27946a1c-eb5b-4b5d-bf13-4b6e4c72fe6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.000594 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27946a1c-eb5b-4b5d-bf13-4b6e4c72fe6b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-27946a1c-eb5b-4b5d-bf13-4b6e4c72fe6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:19.015696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:19.016762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:19.017800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:19.017800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:19.017800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:19.017800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:19.017800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:19.017800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:19.017800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:19.017800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:19.020421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 277/537] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:19.021907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08e8b5-6e62-4d30-9e97-87a68a416ce2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.022378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08e8b5-6e62-4d30-9e97-87a68a416ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.022767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08e8b5-6e62-4d30-9e97-87a68a416ce2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.024483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 278/538] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:19.027602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08e8b5-6e62-4d30-9e97-87a68a416ce2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.046232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe7c3-1622-4735-aef1-0f92216e27f2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.083734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78eb878-67b1-4914-ac91-662ca4c3766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.084252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb878-67b1-4914-ac91-662ca4c3766a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.084582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb878-67b1-4914-ac91-662ca4c3766a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.084856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb878-67b1-4914-ac91-662ca4c3766a None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.085552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78eb878-67b1-4914-ac91-662ca4c3766a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:19.085860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78eb878-67b1-4914-ac91-662ca4c3766a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.087521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78eb878-67b1-4914-ac91-662ca4c3766a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.101573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 261/539] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:19.102970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b793-5762-43d6-8ba9-ddcd81ebf10f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.103459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b793-5762-43d6-8ba9-ddcd81ebf10f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.103709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b793-5762-43d6-8ba9-ddcd81ebf10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.123008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3921b793-5762-43d6-8ba9-ddcd81ebf10f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:19.124524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3921b793-5762-43d6-8ba9-ddcd81ebf10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:19.151177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08e8b5-6e62-4d30-9e97-87a68a416ce2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.152373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.164734 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.197528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.198129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.198405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.198946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.199645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:19.200352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.202042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.208181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 262/540] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:19.209364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b793-5762-43d6-8ba9-ddcd81ebf10f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.218949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 279/541] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:19.227545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.396948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c36ea1-6974-4893-a493-feddc99d62d1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.403486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b793-5762-43d6-8ba9-ddcd81ebf10f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.471105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.472301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.473262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.474061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.474550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.475387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.476441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.476776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=48ad70aa047d443b97e13447397a8a15) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:19.477244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.477672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.479296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.480225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=75aa23cd162e4da2bd6a936c27e51f41) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:19.480703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.482274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.494129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 280/542] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/role_assignments?user.id=48ad70aa047d443b97e13447397a8a15&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:19.495404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.499123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 263/543] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/role_assignments?user.id=75aa23cd162e4da2bd6a936c27e51f41&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:19.536031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.617065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffba86-7188-4e12-8419-f0594e3997b5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.656708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be65b1-e8f4-425e-b16f-64da9506f26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.657693 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be65b1-e8f4-425e-b16f-64da9506f26e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.658144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be65b1-e8f4-425e-b16f-64da9506f26e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.658586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be65b1-e8f4-425e-b16f-64da9506f26e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.671568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be65b1-e8f4-425e-b16f-64da9506f26e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-56be65b1-e8f4-425e-b16f-64da9506f26e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:19.677568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:19.679068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:19.679870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:19.682314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 281/544] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:19.689929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.739815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.826077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a82d6-86ef-445e-9db0-6bce7690380f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.860361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.860968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.861242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.861517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.862164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:19.862462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.863845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.875023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 264/545] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:19.878161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be65b1-e8f4-425e-b16f-64da9506f26e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.994517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.003182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be65b1-e8f4-425e-b16f-64da9506f26e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.044172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.044798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.045070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.045338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.056060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.057507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:20.072959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:20.073875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:20.074683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:20.074683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:20.074683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:20.078003 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 282/546] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:20.080163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.189023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.201379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c30653-0057-420c-afb1-34ab41f75a13 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.236546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.237145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.237419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.237685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.238318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:20.238612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.239998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.257666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 265/547] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:20.259051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.392446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7aa9d2-2859-49a4-a7ce-864160c550bf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.438746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.439403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.439681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.439949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.450382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.451855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:20.467348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:20.468760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:20.469809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 283/548] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:20.475728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.606843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25ba66-8fe3-45ab-9316-0b706d65ee93 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.642164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75d3a4-fe52-4376-89fe-5f185cb7cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.642764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75d3a4-fe52-4376-89fe-5f185cb7cf47 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.643069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75d3a4-fe52-4376-89fe-5f185cb7cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.643334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75d3a4-fe52-4376-89fe-5f185cb7cf47 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.644050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75d3a4-fe52-4376-89fe-5f185cb7cf47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:20.644402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75d3a4-fe52-4376-89fe-5f185cb7cf47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.645861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75d3a4-fe52-4376-89fe-5f185cb7cf47 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.670437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 266/549] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:20.676206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.682387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.703380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c73df7-c7a3-47d0-a6d7-cf4b8d11a2c0 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.703691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c73df7-c7a3-47d0-a6d7-cf4b8d11a2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.703938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c73df7-c7a3-47d0-a6d7-cf4b8d11a2c0 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.705513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 267/550] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:20.710350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0e26-09d9-4a58-8b61-f63cf2df63a3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.710655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0e26-09d9-4a58-8b61-f63cf2df63a3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.710954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a0e26-09d9-4a58-8b61-f63cf2df63a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.725348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.729989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-051a0e26-09d9-4a58-8b61-f63cf2df63a3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:20.731417 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-051a0e26-09d9-4a58-8b61-f63cf2df63a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:20.807708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38431b-46f6-4175-b1dc-dd5dc058af1e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.849021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b40fef-1e2f-42c4-9f66-88a472a208e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.849680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b40fef-1e2f-42c4-9f66-88a472a208e5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.850471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b40fef-1e2f-42c4-9f66-88a472a208e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.850863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b40fef-1e2f-42c4-9f66-88a472a208e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.919046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 268/551] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:20.953341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b40fef-1e2f-42c4-9f66-88a472a208e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.954795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b40fef-1e2f-42c4-9f66-88a472a208e5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.963215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.097688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 284/552] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:21.103043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a0e26-09d9-4a58-8b61-f63cf2df63a3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.220785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b40fef-1e2f-42c4-9f66-88a472a208e5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.284121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a0e26-09d9-4a58-8b61-f63cf2df63a3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.345061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a25d7-090c-41f0-ac8a-b32e8115a272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.346050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a25d7-090c-41f0-ac8a-b32e8115a272 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.346558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a25d7-090c-41f0-ac8a-b32e8115a272 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.347077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a25d7-090c-41f0-ac8a-b32e8115a272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.357904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a25d7-090c-41f0-ac8a-b32e8115a272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.359749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a25d7-090c-41f0-ac8a-b32e8115a272 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-198a25d7-090c-41f0-ac8a-b32e8115a272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:21.375668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:21.376705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:21.377547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:21.377547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:21.377547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:21.378509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b40fef-1e2f-42c4-9f66-88a472a208e5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.379864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 269/553] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:21.381855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9da948-4579-4492-bd72-ee3317ec9d7d None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.382366 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9da948-4579-4492-bd72-ee3317ec9d7d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.382843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9da948-4579-4492-bd72-ee3317ec9d7d None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.384536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 270/554] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:21.388536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.390230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9da948-4579-4492-bd72-ee3317ec9d7d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.392054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.428415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558de29-a533-4e6d-887d-2befe2d4f145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.429052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558de29-a533-4e6d-887d-2befe2d4f145 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.429327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558de29-a533-4e6d-887d-2befe2d4f145 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.429593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558de29-a533-4e6d-887d-2befe2d4f145 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.444155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558de29-a533-4e6d-887d-2befe2d4f145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e558de29-a533-4e6d-887d-2befe2d4f145 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:21.450058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:21.451095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:21.452052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 285/555] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:21.453231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224f388-f2a1-41de-b4f9-2d0c455d0b8e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.453231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224f388-f2a1-41de-b4f9-2d0c455d0b8e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.453564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224f388-f2a1-41de-b4f9-2d0c455d0b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.476605 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c224f388-f2a1-41de-b4f9-2d0c455d0b8e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:21.478004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c224f388-f2a1-41de-b4f9-2d0c455d0b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:21.514641 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9da948-4579-4492-bd72-ee3317ec9d7d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.552130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.553800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.559025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 286/556] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:21.559025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224f388-f2a1-41de-b4f9-2d0c455d0b8e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.559833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.560330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.561191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:21.561698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.567428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.588507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 271/557] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:21.601471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.660633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.666610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224f388-f2a1-41de-b4f9-2d0c455d0b8e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.703167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.703786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.704016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.704285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.704944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:21.705506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.706855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.716421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 287/558] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:21.727391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.732014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.827815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7ffc-e5f9-4dde-9f9a-f6c68229a51e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.880169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.881385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.881715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.881986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.893930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.895609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:21.914001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:21.914983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:21.915897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 272/559] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:21.916837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.922220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cbde9-0be7-4bde-8759-0750cba57d3d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.971617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.972564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.972844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.973550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.986959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.989015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:22.007738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:22.008789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:22.009657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:22.009657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:22.009657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:22.009657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 288/560] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:22.014915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.024806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.040229 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13d02c-7c39-4497-ae19-44e5c425b953 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.049628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.076237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5f616-be2b-4217-8221-a49f94b56332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.076866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f616-be2b-4217-8221-a49f94b56332 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.077140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f616-be2b-4217-8221-a49f94b56332 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.077405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f616-be2b-4217-8221-a49f94b56332 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.087457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5f616-be2b-4217-8221-a49f94b56332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-21b5f616-be2b-4217-8221-a49f94b56332 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:22.092557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:22.093444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:22.094343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 273/561] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:22.096036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5f616-be2b-4217-8221-a49f94b56332 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.134079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e251e-095e-4996-b0b6-342691460d8a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.171238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d5fc1-96a3-46c2-a0cf-4f9745ebb841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.171742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5fc1-96a3-46c2-a0cf-4f9745ebb841 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.172016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5fc1-96a3-46c2-a0cf-4f9745ebb841 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.172283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5fc1-96a3-46c2-a0cf-4f9745ebb841 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.173362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d5fc1-96a3-46c2-a0cf-4f9745ebb841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:22.173362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d5fc1-96a3-46c2-a0cf-4f9745ebb841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.174755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d5fc1-96a3-46c2-a0cf-4f9745ebb841 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.194964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 289/562] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.194964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14733ce6-d42b-42af-a645-2851087ae3ea None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.194964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14733ce6-d42b-42af-a645-2851087ae3ea None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.194964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14733ce6-d42b-42af-a645-2851087ae3ea None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.194964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 290/563] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:22.194964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14733ce6-d42b-42af-a645-2851087ae3ea None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.213958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5f616-be2b-4217-8221-a49f94b56332 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.250408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.250960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.251240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.251507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.252140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:22.252447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.253840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.270086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 274/564] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.270086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.272925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.299356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.310612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14733ce6-d42b-42af-a645-2851087ae3ea None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.344724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1254c-a671-40ab-b424-a1625f9c2517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.345328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1254c-a671-40ab-b424-a1625f9c2517 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.345610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1254c-a671-40ab-b424-a1625f9c2517 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.345885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1254c-a671-40ab-b424-a1625f9c2517 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.346544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1254c-a671-40ab-b424-a1625f9c2517 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:22.346846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1254c-a671-40ab-b424-a1625f9c2517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.348243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1254c-a671-40ab-b424-a1625f9c2517 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.357148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 291/565] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.358777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d5376-9a88-4da8-90f6-5ddc5850d939 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.359429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d5376-9a88-4da8-90f6-5ddc5850d939 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.359787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d5376-9a88-4da8-90f6-5ddc5850d939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.370094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf078ac-1d96-47e1-b248-c85d49359363 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.377051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-645d5376-9a88-4da8-90f6-5ddc5850d939 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:22.378231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-645d5376-9a88-4da8-90f6-5ddc5850d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:22.404912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.405591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.405873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.406146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.406870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=91df91b1ea4949119245c86bf52e32f6) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:22.407208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.408637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.417126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 275/566] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/role_assignments?user.id=91df91b1ea4949119245c86bf52e32f6&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.417926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.458239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 292/567] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:22.459682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9a269-5ee9-4bdf-9505-4e58d7be255a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.459975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9a269-5ee9-4bdf-9505-4e58d7be255a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.460246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9a269-5ee9-4bdf-9505-4e58d7be255a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.465530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 293/568] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:22.465530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9a269-5ee9-4bdf-9505-4e58d7be255a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.544019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.551148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.560055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d12ed-5165-48db-a133-f5a0def66e69 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.604725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9a269-5ee9-4bdf-9505-4e58d7be255a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.604991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd9aa3-e410-4155-875a-bd9aaa0699bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.605814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd9aa3-e410-4155-875a-bd9aaa0699bb None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.606202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd9aa3-e410-4155-875a-bd9aaa0699bb None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.606569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd9aa3-e410-4155-875a-bd9aaa0699bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.614548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd9aa3-e410-4155-875a-bd9aaa0699bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.617512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd9aa3-e410-4155-875a-bd9aaa0699bb None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-11fd9aa3-e410-4155-875a-bd9aaa0699bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:22.629504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:22.630478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:22.632020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:22.635813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 276/569] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:22.637108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef047cb-8f13-41ed-be38-316421fbf682 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.637401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef047cb-8f13-41ed-be38-316421fbf682 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.637682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef047cb-8f13-41ed-be38-316421fbf682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.641207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.641828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.642105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.642373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.653115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.654545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.666530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bef047cb-8f13-41ed-be38-316421fbf682 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:22.668076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bef047cb-8f13-41ed-be38-316421fbf682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:22.669587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:22.670550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:22.672047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:22.672047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:22.672047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:22.675297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 294/570] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:22.676140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.748511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 277/571] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:22.749416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef047cb-8f13-41ed-be38-316421fbf682 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.802320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd0a0d-856a-4f6f-8b4f-09a503d50488 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.816124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.840198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.840793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.841068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.841338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.842062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:22.842364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.843834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.866122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.875706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 295/572] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.875882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.889958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef047cb-8f13-41ed-be38-316421fbf682 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.937972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.939233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.939519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.939789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.940675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:22.940982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.943116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.958261 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 278/573] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.964115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.000134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e255492-3e0f-429a-8937-ca7c340cfd22 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.037164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9484707-3b3d-4d12-b262-dbd2d7538e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.037825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484707-3b3d-4d12-b262-dbd2d7538e87 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.038104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484707-3b3d-4d12-b262-dbd2d7538e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.038376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484707-3b3d-4d12-b262-dbd2d7538e87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.049658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9484707-3b3d-4d12-b262-dbd2d7538e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b9484707-3b3d-4d12-b262-dbd2d7538e87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:23.055144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:23.056305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 296/574] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:23.057268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9484707-3b3d-4d12-b262-dbd2d7538e87 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.075911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cde43-8a82-4d3b-a415-b9331069422f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.092152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.112018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.112651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.112931 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.113202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.124516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.125959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.127764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:23.142737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:23.143743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:23.144605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:23.144605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:23.144605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:23.147373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 279/575] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:23.147965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.176101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9484707-3b3d-4d12-b262-dbd2d7538e87 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.215504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.216138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.216413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.216736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.225308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.226818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:23.239067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:23.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 297/576] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:23.246934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.259135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bf467-32a6-4bfc-a0cc-398de2f41b5e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.291557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee98ffd-fad1-418b-87c1-379f071f1c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.292195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee98ffd-fad1-418b-87c1-379f071f1c87 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.292471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee98ffd-fad1-418b-87c1-379f071f1c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.292738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee98ffd-fad1-418b-87c1-379f071f1c87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.303908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee98ffd-fad1-418b-87c1-379f071f1c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-8ee98ffd-fad1-418b-87c1-379f071f1c87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:23.309130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:23.310242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 280/577] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:23.311207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee98ffd-fad1-418b-87c1-379f071f1c87 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.357985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.370540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.372569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb74941-b5e6-4f25-9ef7-108a29594b83 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.419612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.420230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.420509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.420779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.421444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:23.421627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee98ffd-fad1-418b-87c1-379f071f1c87 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.421726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.423278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.434306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 298/578] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:23.439640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.457387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.458122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.458498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.458860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.459656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:23.460064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.461537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.473377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 281/579] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:23.479155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.548482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d7a3c-2e3d-4bd3-b061-be2d3b1330d6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.585643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb50a3-16a8-4e4b-8122-af3cba43348f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.586946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.587536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.587814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.588083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.588800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:23.589107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.590519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.602355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.608655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 299/580] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:23.609459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.624793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.630143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.631046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.631517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.631859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.632523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:23.632824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.634175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.646849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 282/581] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:23.647714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.718493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fcd7e-6808-472a-83d6-9a408fc7f071 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.753088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf2a59-d99b-47d4-a24a-407674a3f0ef None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.755375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.756029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.756309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.756582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.766582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.768132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:23.785442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:23.786368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:23.787355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:23.787355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:23.787355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:23.787355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 300/582] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:23.787355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.794908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.795782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.796235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.796677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.805252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.806836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:23.819702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:23.820539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:23.821398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 283/583] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:23.821868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.855860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.860452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.898032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a875f46-8b77-4712-b385-bbcd8c48ccab None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.931357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a7a52-ce49-4b21-9a87-3a45e4f719fa None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:23.932233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.932860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.933143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.933416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.944627 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.946046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:23.960925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:23.961860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:23.962720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:23.962720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:23.962720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:23.962720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 301/584] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:23.962980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.973966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:23.974580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.974860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.975177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.985668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:23.987209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:24.001649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:24.002563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:24.008888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:24.008888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:24.008888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:24.008888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 284/585] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:24.008888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.074247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3318637-3ed5-43bb-8c19-6256cab01b45 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.089121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.102523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.113114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.113654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.114027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.114296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.114845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:24.115303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.116099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eeb2a-a240-45d2-8c8b-3ab6ef6136b0 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.116698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.135301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 302/586] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.136417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.153873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.154488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.154771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.155078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.155771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:24.156087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.157510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.169979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 285/587] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.170778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.250341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede83c-72e0-4057-8b3c-8ba4a3cc9209 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.274642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7e457-4a43-45a3-99b4-4aa92042e148 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.286361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.287089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.287291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.287559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.288195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:24.288495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.289832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.300852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 303/588] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.305578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.316568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.317173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.317453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.317728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.319202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:24.319508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.320861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.336956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 286/589] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.336956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.346265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.346808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.420207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb27910-37a6-4368-b764-e279e1c9f470 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.446723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186954dc-79b9-4957-a188-2ace51de7fe9 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.457499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.458177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.458463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.458733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.467413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.468837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.480433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963d387-7044-4a31-943e-edbdf38ddf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:24.480820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:24.481709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:24.487679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:24.488203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963d387-7044-4a31-943e-edbdf38ddf17 None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.488203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963d387-7044-4a31-943e-edbdf38ddf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.488203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963d387-7044-4a31-943e-edbdf38ddf17 None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/152edb19fd7b434485b0e1ebc5078335/roles/1b16ca28088b4fb4a1e9cc15fea47675` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.488388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 304/590] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:24.488388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.502455 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963d387-7044-4a31-943e-edbdf38ddf17 None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=152edb19fd7b434485b0e1ebc5078335, role_id=1b16ca28088b4fb4a1e9cc15fea47675)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.504004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963d387-7044-4a31-943e-edbdf38ddf17 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.530934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 287/591] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:24 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/152edb19fd7b434485b0e1ebc5078335/roles/1b16ca28088b4fb4a1e9cc15fea47675 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:24.530934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963d387-7044-4a31-943e-edbdf38ddf17 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.602361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.612640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.623916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aff86-428a-4d07-abb1-d92e88d8543e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.659121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963d387-7044-4a31-943e-edbdf38ddf17 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.668261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.668906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.669395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.669505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.676954 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.679088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:24.691997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:24.692941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:24.694531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 305/592] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:24.695960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.706119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa8e4e-b800-44a3-8a5b-2c007ddf3015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.706768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8e4e-b800-44a3-8a5b-2c007ddf3015 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.707081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8e4e-b800-44a3-8a5b-2c007ddf3015 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.707743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8e4e-b800-44a3-8a5b-2c007ddf3015 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.716514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa8e4e-b800-44a3-8a5b-2c007ddf3015 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.718129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa8e4e-b800-44a3-8a5b-2c007ddf3015 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-24fa8e4e-b800-44a3-8a5b-2c007ddf3015 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:24.732547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:24.733515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:24.734463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 288/593] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:24.735067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d39b99-0697-4d33-b132-5467f53bde8a None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.735657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d39b99-0697-4d33-b132-5467f53bde8a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.735657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d39b99-0697-4d33-b132-5467f53bde8a None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.737822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 289/594] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:24.738532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d39b99-0697-4d33-b132-5467f53bde8a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.816715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e672bd-f437-41cd-94dd-274d46e806cf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.844508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d39b99-0697-4d33-b132-5467f53bde8a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.854241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.855177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.855467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.855761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.856516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:24.857046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.858502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.862596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.877618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 306/595] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.878234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.885163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d585b8-792c-4c3b-97de-0122d3f632e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.885775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585b8-792c-4c3b-97de-0122d3f632e6 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.886055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585b8-792c-4c3b-97de-0122d3f632e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.886327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585b8-792c-4c3b-97de-0122d3f632e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.887050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d585b8-792c-4c3b-97de-0122d3f632e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:24.887358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d585b8-792c-4c3b-97de-0122d3f632e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.888776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d585b8-792c-4c3b-97de-0122d3f632e6 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.896995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.905551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 290/596] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.905551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70e465-c8be-4b7c-9039-1cd4c26ed444 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.905551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70e465-c8be-4b7c-9039-1cd4c26ed444 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.905551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70e465-c8be-4b7c-9039-1cd4c26ed444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.921456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-cc70e465-c8be-4b7c-9039-1cd4c26ed444 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:24.922584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-cc70e465-c8be-4b7c-9039-1cd4c26ed444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:24.989890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb0f7b-c7aa-4f60-a7cb-83ac0813eb4a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.995494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 291/597] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:24.996331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70e465-c8be-4b7c-9039-1cd4c26ed444 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.043723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.044363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.044641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.044910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.045891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:25.047017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.048507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.072563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 307/598] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:25.073386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.137774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70e465-c8be-4b7c-9039-1cd4c26ed444 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.149533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.164519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.185457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.187083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.187774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.188463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.203322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.205214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.205645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85023-7df3-4cf5-a9a7-a2cd0b9b36ff None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:25.225654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:25.226885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:25.228047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:25.228047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:25.228047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:25.235113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 292/599] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:25.236592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.249563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f1eb-c2ae-4b0a-b93c-f56f618af49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.250391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f1eb-c2ae-4b0a-b93c-f56f618af49b None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.250766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f1eb-c2ae-4b0a-b93c-f56f618af49b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.251176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f1eb-c2ae-4b0a-b93c-f56f618af49b None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/c6774fdd8b2e46b6a7d00919568f9fd1/roles/1b16ca28088b4fb4a1e9cc15fea47675` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.267529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62f1eb-c2ae-4b0a-b93c-f56f618af49b None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=c6774fdd8b2e46b6a7d00919568f9fd1, role_id=1b16ca28088b4fb4a1e9cc15fea47675)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.269169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62f1eb-c2ae-4b0a-b93c-f56f618af49b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.285650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 308/600] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:25 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/c6774fdd8b2e46b6a7d00919568f9fd1/roles/1b16ca28088b4fb4a1e9cc15fea47675 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:25.286675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f1eb-c2ae-4b0a-b93c-f56f618af49b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.361124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896d972-1674-4721-aa3a-5a187de53ac5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.416061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.417081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.417358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.417626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.435815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.439279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.442966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f1eb-c2ae-4b0a-b93c-f56f618af49b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:25.464856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:25.465843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:25.466706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:25.466706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:25.466706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:25.469264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 293/601] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:25.470293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.471396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.489134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.492476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.493174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.493349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.493618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.494266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:25.494574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.496030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.513248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 309/602] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:25.514325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.592321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc863-ac61-45c4-9a92-32d361f6c951 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.631424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569ce6-2544-4492-87d0-607156e195a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.632433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569ce6-2544-4492-87d0-607156e195a3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.632928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569ce6-2544-4492-87d0-607156e195a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.633433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569ce6-2544-4492-87d0-607156e195a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.637229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4076e-5519-426e-920d-dcdd182574ce None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.644324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2569ce6-2544-4492-87d0-607156e195a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-c2569ce6-2544-4492-87d0-607156e195a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:25.649848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:25.650778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:25.651636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:25.658728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 294/603] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:25.659857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569ce6-2544-4492-87d0-607156e195a3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.675482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.676021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.676297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.676563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.677519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:25.677519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.679517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.694536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 310/604] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:25.699457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.737294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.754445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.773005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569ce6-2544-4492-87d0-607156e195a3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.822881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.823692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df599-b96a-4ab1-ad82-991aba42b0a4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.823834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.823968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.824242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.838541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.840779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:25.856911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:25.857924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:25.858791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:25.858791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:25.858791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:25.858791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:25.858791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:25.858791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:25.858791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:25.858791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:25.862925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 295/605] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:25.863934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.864600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.865302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.865713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.866087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.866862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:25.867251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.868663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.878378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 311/606] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:25.879242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.989138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2eec9a-a944-43e8-ae22-3d8f6ef34047 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.991325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.993912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba808df4-3b20-4f71-9d02-86f5f3577257 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.029834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.034552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.035028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.035682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.035911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.036020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.036214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.036685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.037955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.039666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:26.040711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.042433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.044663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.046208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.056104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 312/607] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:26.057207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:26.059420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:26.060280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:26.061231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:26.061231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:26.061231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:26.061231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:26.061231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:26.061231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:26.061231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:26.061231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:26.068652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 296/608] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:26.068652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.183730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9526a7-342b-483b-ae25-636658ac8cb8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.204601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce9df-83c9-4e41-a81c-3403b280aa09 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.219612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.220258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.220537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.220808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.233582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.235088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:26.251323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:26.252255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:26.253133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:26.253133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:26.253133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:26.253133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 313/609] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:26.253765 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.254392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcb161-8a86-4df9-9a2e-7d074beeb77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.255306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcb161-8a86-4df9-9a2e-7d074beeb77d None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.255828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcb161-8a86-4df9-9a2e-7d074beeb77d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.256567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcb161-8a86-4df9-9a2e-7d074beeb77d None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/75aa23cd162e4da2bd6a936c27e51f41/roles/1b16ca28088b4fb4a1e9cc15fea47675` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.272993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.278042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcb161-8a86-4df9-9a2e-7d074beeb77d None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=75aa23cd162e4da2bd6a936c27e51f41, role_id=1b16ca28088b4fb4a1e9cc15fea47675)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.280848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcb161-8a86-4df9-9a2e-7d074beeb77d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.300137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 297/610] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:26 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/75aa23cd162e4da2bd6a936c27e51f41/roles/1b16ca28088b4fb4a1e9cc15fea47675 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:26.301233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcb161-8a86-4df9-9a2e-7d074beeb77d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.417502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.446067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f4ab7-cfd0-4d5d-a389-6f7b9fbc2d8b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.494964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.494964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.494964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.494964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.495411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:26.495411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.495411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.521291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 314/611] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:26.523271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcb161-8a86-4df9-9a2e-7d074beeb77d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.528071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.563396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.564016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.564296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.564566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.565204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:26.565507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.567688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.580024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 298/612] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:26.585262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.619775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.648106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e039-654d-42e1-97e7-da3b10691b4d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.689980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fb07e-bb6c-4162-aac0-6d11741729a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.690645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb07e-bb6c-4162-aac0-6d11741729a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.691050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb07e-bb6c-4162-aac0-6d11741729a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.691457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb07e-bb6c-4162-aac0-6d11741729a1 None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/48ad70aa047d443b97e13447397a8a15/roles/1b16ca28088b4fb4a1e9cc15fea47675` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.709577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fb07e-bb6c-4162-aac0-6d11741729a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=48ad70aa047d443b97e13447397a8a15, role_id=1b16ca28088b4fb4a1e9cc15fea47675)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.711340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fb07e-bb6c-4162-aac0-6d11741729a1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.722167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309a6ef-8fa8-4ee6-81ba-513ffdd37716 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.724056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.729648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 315/613] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:26 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/48ad70aa047d443b97e13447397a8a15/roles/1b16ca28088b4fb4a1e9cc15fea47675 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:46:26.767049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f069ed-8d50-4491-aa4c-7113e59a14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.767702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f069ed-8d50-4491-aa4c-7113e59a14ae None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.767980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f069ed-8d50-4491-aa4c-7113e59a14ae None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.768254 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f069ed-8d50-4491-aa4c-7113e59a14ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.777620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f069ed-8d50-4491-aa4c-7113e59a14ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.779185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f069ed-8d50-4491-aa4c-7113e59a14ae None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-88f069ed-8d50-4491-aa4c-7113e59a14ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:26.796545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:26.797548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:26.798977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 299/614] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:26.806284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fb07e-bb6c-4162-aac0-6d11741729a1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.949190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2b81e-a201-480d-949a-136ce549c3da None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.949532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2b81e-a201-480d-949a-136ce549c3da None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.949740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2b81e-a201-480d-949a-136ce549c3da None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.951468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 300/615] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:26.956688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920caa9-f26e-46e4-b59a-a2e6425ed183 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.956973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920caa9-f26e-46e4-b59a-a2e6425ed183 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.957246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920caa9-f26e-46e4-b59a-a2e6425ed183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.959603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fb07e-bb6c-4162-aac0-6d11741729a1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.976975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9920caa9-f26e-46e4-b59a-a2e6425ed183 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:26.978233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9920caa9-f26e-46e4-b59a-a2e6425ed183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:27.007722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:27.026876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.027554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.027838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.028054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.028815 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:27.029117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.030572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:27.051518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 316/616] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:27.057397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.081127 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 301/617] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:27.185243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c91356-1bb5-4ef0-8812-b78f28a7d345 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.226332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.227309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.227784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.228224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.245156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.246864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:27.266063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:27.271883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:27.272944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:27.272944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:27.272944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:27.274617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 317/618] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:27.280292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920caa9-f26e-46e4-b59a-a2e6425ed183 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.436837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:27.450636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920caa9-f26e-46e4-b59a-a2e6425ed183 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.493749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.494391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.494676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.495045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.495714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:27.496019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.497424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:27.500492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.510758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 302/619] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:27.516061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.563871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:27.686017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae8f2e-d7d9-42cc-9dea-fbed13207ea3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.706284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd52d6-d5bf-4a1f-93a6-5f5648b87135 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.749375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86c578-a114-452d-8838-078c7bda215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.750337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86c578-a114-452d-8838-078c7bda215f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.750513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.751161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86c578-a114-452d-8838-078c7bda215f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.751419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.751765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.751990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86c578-a114-452d-8838-078c7bda215f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.752088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.760462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.762651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:27.764899 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86c578-a114-452d-8838-078c7bda215f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3f86c578-a114-452d-8838-078c7bda215f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:27.770654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:27.771643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:27.772470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:27.774626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:27.775586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:27.776450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:27.776450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:27.776450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:27.776450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:27.776450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:27.776450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:27.776450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:27.776450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:27.777810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 318/620] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:27.782991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 303/621] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:27.782991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.784061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c15033-c787-4ae9-ab9d-04563156dd87 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.784343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c15033-c787-4ae9-ab9d-04563156dd87 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.784614 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c15033-c787-4ae9-ab9d-04563156dd87 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.786138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 319/622] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:27.786880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c15033-c787-4ae9-ab9d-04563156dd87 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.799823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:27.893443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:27.902442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f3f57-4eac-43ae-af49-ad61596e79e1 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.905997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c15033-c787-4ae9-ab9d-04563156dd87 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.940991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaef843-7bce-4dea-a40d-5f96c2455d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.941424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaef843-7bce-4dea-a40d-5f96c2455d86 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.941704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaef843-7bce-4dea-a40d-5f96c2455d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.941975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaef843-7bce-4dea-a40d-5f96c2455d86 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.942659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaef843-7bce-4dea-a40d-5f96c2455d86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:27.943041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaef843-7bce-4dea-a40d-5f96c2455d86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.944454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaef843-7bce-4dea-a40d-5f96c2455d86 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:27.947175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.948039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.948528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.948985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.949862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:27.950352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.951960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:27.961592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 304/623] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:27.961592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da6b25-f4e3-4339-acf1-4de14736ccbe None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.961592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da6b25-f4e3-4339-acf1-4de14736ccbe None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.961592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da6b25-f4e3-4339-acf1-4de14736ccbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.967579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 320/624] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:27.969025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.978776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-94da6b25-f4e3-4339-acf1-4de14736ccbe None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:27.980132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-94da6b25-f4e3-4339-acf1-4de14736ccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:28.074916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 305/625] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:28.074916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da6b25-f4e3-4339-acf1-4de14736ccbe None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.074916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:28.092713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8293b-4c81-4e43-ac8e-f1fabfff815f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.148013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89af92b-8e46-46ff-a576-bacd72a3284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.148596 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89af92b-8e46-46ff-a576-bacd72a3284a None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.148878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89af92b-8e46-46ff-a576-bacd72a3284a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.149152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89af92b-8e46-46ff-a576-bacd72a3284a None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/91df91b1ea4949119245c86bf52e32f6/roles/1b16ca28088b4fb4a1e9cc15fea47675` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.164633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89af92b-8e46-46ff-a576-bacd72a3284a None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=91df91b1ea4949119245c86bf52e32f6, role_id=1b16ca28088b4fb4a1e9cc15fea47675)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:28.166446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89af92b-8e46-46ff-a576-bacd72a3284a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.181819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 321/626] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:27 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/91df91b1ea4949119245c86bf52e32f6/roles/1b16ca28088b4fb4a1e9cc15fea47675 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:28.235591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da6b25-f4e3-4339-acf1-4de14736ccbe None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.279972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.280611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.281136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.281136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.293871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:28.295444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:28.319776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:28.320765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:28.321640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:28.321640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:28.321640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:28.321640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 306/627] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:28.326651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89af92b-8e46-46ff-a576-bacd72a3284a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.336623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:28.413355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.454916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89af92b-8e46-46ff-a576-bacd72a3284a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.500593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.501213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.501493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.501763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.502418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:28.502719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:28.508656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.523261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 322/628] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:28.529385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.556425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:28.614064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:28.617193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6583b58-0bb8-4ebe-8c2f-cc395c1e6119 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.658669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365fb35-7b88-48d4-af17-2c399d058953 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.664813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e975b-6522-4e87-9bf0-058a89eb6305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.665567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e975b-6522-4e87-9bf0-058a89eb6305 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.665942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e975b-6522-4e87-9bf0-058a89eb6305 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.666314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e975b-6522-4e87-9bf0-058a89eb6305 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.680721 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e975b-6522-4e87-9bf0-058a89eb6305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-a59e975b-6522-4e87-9bf0-058a89eb6305 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:28.687478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:28.688749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:28.689559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 307/629] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:28.699843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e975b-6522-4e87-9bf0-058a89eb6305 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.732164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.732821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.733103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.733370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.754266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:28.756284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:28.784050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:28.785067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:28.786015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:28.789786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 323/630] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:28.795980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.814714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e975b-6522-4e87-9bf0-058a89eb6305 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.843286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:28.882994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.883602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.883880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.884151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.884816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:28.885109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:28.886482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.905347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 308/631] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:28.911542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.991937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75db11-8cd2-45b0-89e8-28cb3b36a3b0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.022729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.032369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303a6f5-8865-4a66-b83c-4ceb0953eb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.033045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a6f5-8865-4a66-b83c-4ceb0953eb00 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.033334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a6f5-8865-4a66-b83c-4ceb0953eb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.033586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a6f5-8865-4a66-b83c-4ceb0953eb00 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.034355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303a6f5-8865-4a66-b83c-4ceb0953eb00 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:29.034661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303a6f5-8865-4a66-b83c-4ceb0953eb00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.036143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303a6f5-8865-4a66-b83c-4ceb0953eb00 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.043535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464f570-d7d1-4526-abbb-05b9f7bfd24a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.056875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 324/632] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:29.058287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a44f4-6405-4b5c-8679-985ed3ddda87 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.058578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a44f4-6405-4b5c-8679-985ed3ddda87 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.058849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a44f4-6405-4b5c-8679-985ed3ddda87 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.064522 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 325/633] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:29.065335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a44f4-6405-4b5c-8679-985ed3ddda87 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.086934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a6c18-bc47-49a8-80a9-501427518e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.087920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a6c18-bc47-49a8-80a9-501427518e8b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.088396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a6c18-bc47-49a8-80a9-501427518e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.089392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a6c18-bc47-49a8-80a9-501427518e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.104007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a6c18-bc47-49a8-80a9-501427518e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.106954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a6c18-bc47-49a8-80a9-501427518e8b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-4f6a6c18-bc47-49a8-80a9-501427518e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:29.125593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:29.126595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:29.127514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:29.127514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:29.127514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:29.135287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 309/634] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:29.136894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7720b-de0e-41ac-a114-74f2810b662d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.138410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7720b-de0e-41ac-a114-74f2810b662d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.138584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7720b-de0e-41ac-a114-74f2810b662d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.161234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-92f7720b-de0e-41ac-a114-74f2810b662d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:29.163217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-92f7720b-de0e-41ac-a114-74f2810b662d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:29.187304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.197029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a44f4-6405-4b5c-8679-985ed3ddda87 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.238271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093bf5b-b07c-4209-8c01-81cd3960278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.239137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093bf5b-b07c-4209-8c01-81cd3960278a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.239586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093bf5b-b07c-4209-8c01-81cd3960278a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.240059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093bf5b-b07c-4209-8c01-81cd3960278a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.243430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 310/635] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:29.244841 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7720b-de0e-41ac-a114-74f2810b662d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.302678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.321237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093bf5b-b07c-4209-8c01-81cd3960278a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.322928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093bf5b-b07c-4209-8c01-81cd3960278a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.354241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7720b-de0e-41ac-a114-74f2810b662d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.399188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bd3fc-fb51-417c-8fd1-eed0bdb79f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.400069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd3fc-fb51-417c-8fd1-eed0bdb79f22 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.400514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd3fc-fb51-417c-8fd1-eed0bdb79f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.400924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd3fc-fb51-417c-8fd1-eed0bdb79f22 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.401851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bd3fc-fb51-417c-8fd1-eed0bdb79f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:29.402855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bd3fc-fb51-417c-8fd1-eed0bdb79f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.405401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bd3fc-fb51-417c-8fd1-eed0bdb79f22 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.426135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 311/636] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:29.428856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22fd3f-1726-4ff0-a2d0-1bfe8d5059a4 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.429386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22fd3f-1726-4ff0-a2d0-1bfe8d5059a4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.429797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22fd3f-1726-4ff0-a2d0-1bfe8d5059a4 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.431903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 312/637] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:29.432766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22fd3f-1726-4ff0-a2d0-1bfe8d5059a4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.449798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 326/638] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:29.451301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d479-4b58-4a26-bae8-4e0d7966d731 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.451686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d479-4b58-4a26-bae8-4e0d7966d731 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.452073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d479-4b58-4a26-bae8-4e0d7966d731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.457667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.476214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5374d479-4b58-4a26-bae8-4e0d7966d731 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:29.478135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5374d479-4b58-4a26-bae8-4e0d7966d731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:29.552344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22fd3f-1726-4ff0-a2d0-1bfe8d5059a4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.559090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 327/639] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:29.559090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d479-4b58-4a26-bae8-4e0d7966d731 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.579331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.596199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.597552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.598280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.598770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.609310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.611753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:29.628822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:29.629808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:29.630982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:29.634360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 313/640] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:29.635370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.680637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d479-4b58-4a26-bae8-4e0d7966d731 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.721920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.722540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.722823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.723115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.734576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.736474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.743239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:29.759315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:29.760282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:29.761174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:29.761174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:29.761174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:29.765246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f41573-1203-498d-ad00-bc99805cc880 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.765671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 328/641] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:29.766515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.800019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f1f41-6396-465a-a1a8-43d796ac3211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.800639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f1f41-6396-465a-a1a8-43d796ac3211 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.800916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f1f41-6396-465a-a1a8-43d796ac3211 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.801182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f1f41-6396-465a-a1a8-43d796ac3211 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.812384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f1f41-6396-465a-a1a8-43d796ac3211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-e79f1f41-6396-465a-a1a8-43d796ac3211 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:29.817892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:29.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:29.820343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 314/642] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:29.821618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f1f41-6396-465a-a1a8-43d796ac3211 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.845791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.882267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5108d-ee10-4019-982b-2fc37fb4c086 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.920039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.920636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.920913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.921181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.921677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f1f41-6396-465a-a1a8-43d796ac3211 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.921900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:29.922199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.923814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.946036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 329/643] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:29.947019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.961181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.962537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.963308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.963811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.964907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:29.965783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.967347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.977783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.989646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 315/644] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:29.989646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.054111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5b01b-c2a2-46f1-99a6-3253f8584be6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.079106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.092000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.092611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.092894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.093168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.093828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:30.094131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.094673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd0abc-8edc-4feb-ad6e-40c0a90cec79 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.095515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.104630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 330/645] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:30.106262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.134499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a9be-1d71-4ebc-81da-e87238c42984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.135456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a9be-1d71-4ebc-81da-e87238c42984 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.135456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a9be-1d71-4ebc-81da-e87238c42984 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.135653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a9be-1d71-4ebc-81da-e87238c42984 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.145899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3a9be-1d71-4ebc-81da-e87238c42984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-50d3a9be-1d71-4ebc-81da-e87238c42984 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:30.151183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:30.152101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 316/646] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:30.153081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a9be-1d71-4ebc-81da-e87238c42984 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.219539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.222934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3031476-74f1-42ea-b1e6-712ee5dfee4f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.257844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cf915-ce62-4506-b626-01678f672a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.258597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cf915-ce62-4506-b626-01678f672a0c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.259020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cf915-ce62-4506-b626-01678f672a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.259367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cf915-ce62-4506-b626-01678f672a0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.265498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a9be-1d71-4ebc-81da-e87238c42984 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.300533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.301402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.301679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.301947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.308757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.310220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:30.321869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:30.322982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:30.323881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 317/647] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:30.324369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.324998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.342589 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cf915-ce62-4506-b626-01678f672a0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.344279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cf915-ce62-4506-b626-01678f672a0c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.434145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de45b4b-e49f-45c1-8c69-756ce2c4f8ed None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.452675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.460251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 331/648] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:30.460251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cf915-ce62-4506-b626-01678f672a0c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.475852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.476471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.476756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.477023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.488318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.489750 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:30.504291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:30.510580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:30.511873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:30.511873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:30.511873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:30.511873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 318/649] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:30.511873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.562023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.578850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cf915-ce62-4506-b626-01678f672a0c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.616691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.617470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.617573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5808f5f-9305-4553-ad07-679b7dfdd732 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.617920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.618291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.619118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:30.619523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.621019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.635386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 332/650] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:30.635386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.657341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.658040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.658389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.658731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.659524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:30.659890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.661379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.673583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 319/651] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:30.674232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.704237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.749466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d390f-2068-414e-b7f8-1e7bc8ca1965 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.773425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72ac4c-5704-4e7e-ab4b-314e682c3a1f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.782501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.783159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.783439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.783715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.794433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.795886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.804735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.809598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.810202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.810522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.810876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.811463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:30.811722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:30.812673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:30.813823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:30.813823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:30.813823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:30.814002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.814076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 333/652] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:30.814152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.814227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.825166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 320/653] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:30.825962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.914759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.918029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d62ab-db80-4452-986f-8672f5792db2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.924902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4ccc6-41c7-447c-8c56-d4021c5923a7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.951591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.952296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.952652 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.952996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.957751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320668a6-2b94-487c-b43d-6dafa6ac6ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.958348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320668a6-2b94-487c-b43d-6dafa6ac6ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.958678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320668a6-2b94-487c-b43d-6dafa6ac6ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.958980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320668a6-2b94-487c-b43d-6dafa6ac6ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.959695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320668a6-2b94-487c-b43d-6dafa6ac6ad0 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=152edb19fd7b434485b0e1ebc5078335) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:30.960005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320668a6-2b94-487c-b43d-6dafa6ac6ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.961433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320668a6-2b94-487c-b43d-6dafa6ac6ad0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.963608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.965027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.967189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 321/654] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/role_assignments?user.id=152edb19fd7b434485b0e1ebc5078335&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:30.968316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a97802-1352-4353-b358-f458415c6f98 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.968597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a97802-1352-4353-b358-f458415c6f98 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.968868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a97802-1352-4353-b358-f458415c6f98 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.970206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 322/655] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:30.975674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a97802-1352-4353-b358-f458415c6f98 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:30.980415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:30.981419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:30.982297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:30.982297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:30.982297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:30.986494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 334/656] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:30.987380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.046531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.098433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a97802-1352-4353-b358-f458415c6f98 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.102575 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a509b-1641-4e76-a47f-3b4fc3f34a4b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.136429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45dc842-5f3a-4722-9cd6-30eb228470db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.137045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dc842-5f3a-4722-9cd6-30eb228470db None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.137981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dc842-5f3a-4722-9cd6-30eb228470db None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.138269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dc842-5f3a-4722-9cd6-30eb228470db None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.138940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dc842-5f3a-4722-9cd6-30eb228470db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:31.139245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dc842-5f3a-4722-9cd6-30eb228470db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.140612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dc842-5f3a-4722-9cd6-30eb228470db None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.140716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.141241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.141520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.141790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.152648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.154116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.155085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 323/657] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:31.158963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d13114-9249-4ca1-80ef-8e128c387392 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.158963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d13114-9249-4ca1-80ef-8e128c387392 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.158963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d13114-9249-4ca1-80ef-8e128c387392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:31.166871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:31.167826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:31.168799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 335/658] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:31.169396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.178079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a9d13114-9249-4ca1-80ef-8e128c387392 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:31.179756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a9d13114-9249-4ca1-80ef-8e128c387392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:31.190962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.255733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 324/659] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:31.256720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d13114-9249-4ca1-80ef-8e128c387392 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.292839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a5662-4a88-4c6e-85e1-b59657b58af7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.310341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.340892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.341649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.342443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.342443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.343142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:31.343505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.344931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.363347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 336/660] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:31.364331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.379940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d13114-9249-4ca1-80ef-8e128c387392 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.435355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.436940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.437427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.437903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.445893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.448057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:31.460045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:31.460981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:31.461871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:31.461871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:31.461871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:31.461871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:31.461871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:31.461871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:31.461871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:31.461871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:31.464230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 325/661] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:31.469810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.473761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.486154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124c2d8-45ec-4f78-b679-39a437d931e6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.523153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.523763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.524037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.524303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.525141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:31.525444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.527133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.546087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 337/662] 10.222.0.52 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:31.547285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.580260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.584023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8bb49-0589-4107-9195-8caad9eade64 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.619788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.620414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.620724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.621016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.629247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.631264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:31.642279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:31.643201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 326/663] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:31.644207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.659230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130058e-d45a-4888-a614-c7da75798017 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.699942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.700602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.700879 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.701149 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.701859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:31.702159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.703618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.712904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.716526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 338/664] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:31.717332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.760889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186165c6-6afb-49f6-9edb-ceb6a6e984f2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.799582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11800c-19ea-40e6-8285-ff35c370466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.800122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11800c-19ea-40e6-8285-ff35c370466c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.800397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11800c-19ea-40e6-8285-ff35c370466c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.800685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11800c-19ea-40e6-8285-ff35c370466c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.812163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11800c-19ea-40e6-8285-ff35c370466c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:31.814454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-4a11800c-19ea-40e6-8285-ff35c370466c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:31.818454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:31.819496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 327/665] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:31.820519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11800c-19ea-40e6-8285-ff35c370466c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.829279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce13fec-2b8c-408e-9816-d8763d9f2efe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.864641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605feb-632b-4afe-9597-4a4c54e5fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.865197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605feb-632b-4afe-9597-4a4c54e5fe6b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.865477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605feb-632b-4afe-9597-4a4c54e5fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.865748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605feb-632b-4afe-9597-4a4c54e5fe6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.931712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11800c-19ea-40e6-8285-ff35c370466c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.939289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c605feb-632b-4afe-9597-4a4c54e5fe6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.940737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c605feb-632b-4afe-9597-4a4c54e5fe6b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.955844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.967937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.968532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.968805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.969068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.969788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:31.970084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.971519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.991834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 328/666] 10.222.0.52 () {56 vars in 1139 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:31.992919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.040696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.055940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 339/667] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:32.055940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605feb-632b-4afe-9597-4a4c54e5fe6b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.101157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae40fe4-fe50-4443-9bdb-c7876de5d595 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.137391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.137928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.138209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.138482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.139243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=c6774fdd8b2e46b6a7d00919568f9fd1) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:32.139538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.140959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.156572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 329/668] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/role_assignments?user.id=c6774fdd8b2e46b6a7d00919568f9fd1&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.156572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.167038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605feb-632b-4afe-9597-4a4c54e5fe6b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.205048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.205959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.206248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.206524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.207256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:32.207618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.209610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.210395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.220493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 340/669] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.221443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.284957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.298096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85fd8e-55cf-4f0f-b641-a2bfff5c388b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.348126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de024fe-701a-4ed3-a385-12d54ca2360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.348739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de024fe-701a-4ed3-a385-12d54ca2360e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.349016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de024fe-701a-4ed3-a385-12d54ca2360e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.349865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de024fe-701a-4ed3-a385-12d54ca2360e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.351173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056b495-1a99-4c88-943a-f936ec4b3c2c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.406994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.407906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.408384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.409025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.425304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.426741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.442575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de024fe-701a-4ed3-a385-12d54ca2360e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.444639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de024fe-701a-4ed3-a385-12d54ca2360e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:32.444772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:32.445626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:32.451251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:32.451251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:32.451251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:32.451251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 341/670] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:32.451251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.504700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.571253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdbe7-ffc6-43eb-9dac-129b15c12427 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.574986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 330/671] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:32.575859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de024fe-701a-4ed3-a385-12d54ca2360e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.579890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.619740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.620384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.620668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.620941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.633887 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.636058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:32.654211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:32.656163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:32.658432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:32.658432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:32.658432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:32.666929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 342/672] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:32.666929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.706918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de024fe-701a-4ed3-a385-12d54ca2360e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.787301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.787301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.787301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.787301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.787770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:32.787770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.790911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.820505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 331/673] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.821616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.830916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c0e7-34a6-4843-8305-a1e2732b8bb3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.846911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.878915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.878915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.878915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.878915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.894912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.894912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:32.910313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:32.912013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:32.913564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:32.913564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:32.913564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:32.913564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 343/674] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:32.913564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.928349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.049056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52705d03-3668-4111-9e78-efcab9171255 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.066712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d65c5-985e-4e0a-bbaa-6e11c0a8bc37 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.108448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.109264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.109648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.110010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.110772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.111212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.113061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.128462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 332/675] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.129523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.161740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.167388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.168274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.168696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.186529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.191657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:33.225747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:33.226840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 344/676] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:33.227974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.234049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.281343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9c046-1d81-40d9-8eaf-99065b9a2cd7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.301199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.316547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b1660-dc41-4083-92d1-f05c0aa1de06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.317397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b1660-dc41-4083-92d1-f05c0aa1de06 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.317577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b1660-dc41-4083-92d1-f05c0aa1de06 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.317788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b1660-dc41-4083-92d1-f05c0aa1de06 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.319051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b1660-dc41-4083-92d1-f05c0aa1de06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.319355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b1660-dc41-4083-92d1-f05c0aa1de06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.320764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b1660-dc41-4083-92d1-f05c0aa1de06 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.337109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 333/677] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.337109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4443-5e08-4db1-923d-a060d8f3b3bd None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.337109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4443-5e08-4db1-923d-a060d8f3b3bd None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.337109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4443-5e08-4db1-923d-a060d8f3b3bd None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.337109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 334/678] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:33.337109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d4443-5e08-4db1-923d-a060d8f3b3bd None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.346671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34904f-837b-4872-9adb-71d8762dcab4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.384327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.385519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.386025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.386495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.398134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.400616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:33.414114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:33.415224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:33.416162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:33.420672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 345/679] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:33.421538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.443934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d4443-5e08-4db1-923d-a060d8f3b3bd None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.474065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.483327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac069752-304e-487d-9768-d9ca88e1e242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.484050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac069752-304e-487d-9768-d9ca88e1e242 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.484369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac069752-304e-487d-9768-d9ca88e1e242 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.484644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac069752-304e-487d-9768-d9ca88e1e242 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.485336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac069752-304e-487d-9768-d9ca88e1e242 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.485647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac069752-304e-487d-9768-d9ca88e1e242 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.487110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac069752-304e-487d-9768-d9ca88e1e242 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.503558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 335/680] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.503558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fbd4f-2064-44a6-bb3f-90946d2dc8db None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.503558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fbd4f-2064-44a6-bb3f-90946d2dc8db None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.503558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fbd4f-2064-44a6-bb3f-90946d2dc8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.526210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ab9fbd4f-2064-44a6-bb3f-90946d2dc8db None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:33.527446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ab9fbd4f-2064-44a6-bb3f-90946d2dc8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:33.530211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b790d6-0765-4a03-a2e5-3ac50fce4b6f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.559804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.571487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.572121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.572727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.573003 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.580813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.582248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:33.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:33.594521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 346/681] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:33.595536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.602933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 336/682] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:33.602933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fbd4f-2064-44a6-bb3f-90946d2dc8db None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.715568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b728e3-cfb7-4f92-9ccb-7734a2daedab None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.743018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fbd4f-2064-44a6-bb3f-90946d2dc8db None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.757612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.758376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.758655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.758955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.759809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.760113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.761522 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.762462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.781732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.782617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.783093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.783527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.784400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=75aa23cd162e4da2bd6a936c27e51f41) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.784852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.786411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.787552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 347/683] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.788746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.793451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 337/684] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/role_assignments?user.id=75aa23cd162e4da2bd6a936c27e51f41&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.794423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.846870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.924086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66011b53-9b73-4c13-bef5-e0a0168ca184 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.924324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b172d35-055b-4cf1-9b21-28d3c8925a8f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.964881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c956d-4ea6-4878-a474-001fad3523b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.965779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c956d-4ea6-4878-a474-001fad3523b8 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.966063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c956d-4ea6-4878-a474-001fad3523b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.966402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c956d-4ea6-4878-a474-001fad3523b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.968755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.969725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.970070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.970411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.971349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.971661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.973107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.990389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 348/685] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.990389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.030197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:34.053543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c956d-4ea6-4878-a474-001fad3523b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.055038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c956d-4ea6-4878-a474-001fad3523b8 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.109547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e59520-d8eb-4d2b-8edd-e3461d0ff9be None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.119191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:34.156238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66289d-29c3-466f-ada1-782d8436037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.157747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66289d-29c3-466f-ada1-782d8436037e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.158312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66289d-29c3-466f-ada1-782d8436037e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.158814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66289d-29c3-466f-ada1-782d8436037e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.173350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66289d-29c3-466f-ada1-782d8436037e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-af66289d-29c3-466f-ada1-782d8436037e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:34.181285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:34.182413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:34.183462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:34.184235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 338/686] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:34.184910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c956d-4ea6-4878-a474-001fad3523b8 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.185885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 349/687] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:34.186800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66289d-29c3-466f-ada1-782d8436037e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.273884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:34.296051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c956d-4ea6-4878-a474-001fad3523b8 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.325074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66289d-29c3-466f-ada1-782d8436037e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.332663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.333432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.334391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.334739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.336039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=48ad70aa047d443b97e13447397a8a15) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:34.336420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.338304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.346237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 339/688] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/role_assignments?user.id=48ad70aa047d443b97e13447397a8a15&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:34.347480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.369172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.370726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.371431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.371883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.399687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.401166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:34.430288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:34.431359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:34.432263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:34.432263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:34.432263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:34.434604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 350/689] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:34.441694 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.495316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:34.548582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba5126-2f2b-4aac-ba79-203dc5fff7e7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.602690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c15aa8-3a04-45fc-a315-e60742ef6068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.603343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c15aa8-3a04-45fc-a315-e60742ef6068 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.603625 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c15aa8-3a04-45fc-a315-e60742ef6068 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.603893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c15aa8-3a04-45fc-a315-e60742ef6068 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.604551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c15aa8-3a04-45fc-a315-e60742ef6068 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:34.604851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c15aa8-3a04-45fc-a315-e60742ef6068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.606214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c15aa8-3a04-45fc-a315-e60742ef6068 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.615690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 340/690] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:34.616875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1045f-e8e9-4d5a-ac63-fc0bbcb65480 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.617156 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1045f-e8e9-4d5a-ac63-fc0bbcb65480 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.617487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1045f-e8e9-4d5a-ac63-fc0bbcb65480 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.618946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf221618-488a-4deb-b410-4ab374d07864 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.623072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 341/691] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:34.627838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1045f-e8e9-4d5a-ac63-fc0bbcb65480 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.638101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:34.661225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd769cc1-055a-463b-bd92-7794281fb05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.662368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd769cc1-055a-463b-bd92-7794281fb05d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.662645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd769cc1-055a-463b-bd92-7794281fb05d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.662937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd769cc1-055a-463b-bd92-7794281fb05d None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.663598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd769cc1-055a-463b-bd92-7794281fb05d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:34.663899 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd769cc1-055a-463b-bd92-7794281fb05d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.665483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd769cc1-055a-463b-bd92-7794281fb05d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.683076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 351/692] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:34.684426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b8002-107f-47b2-b162-7b0d2f9a78a3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.684717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b8002-107f-47b2-b162-7b0d2f9a78a3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.684988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b8002-107f-47b2-b162-7b0d2f9a78a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.709259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-aa3b8002-107f-47b2-b162-7b0d2f9a78a3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:34.711082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aa3b8002-107f-47b2-b162-7b0d2f9a78a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:34.747281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1045f-e8e9-4d5a-ac63-fc0bbcb65480 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.783231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.783867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.784147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.784420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.787270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 352/693] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:34.788281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b8002-107f-47b2-b162-7b0d2f9a78a3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.807194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.808609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.833136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:34.838312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:34.839522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:34.840419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:34.840419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:34.840419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:34.842977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 342/694] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:34.846498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.901340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b8002-107f-47b2-b162-7b0d2f9a78a3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.936100 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.936726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.937002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.937274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.946601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.946601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:34.957711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:34.959383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:34.960504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:34.960504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:34.960504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:34.960504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:34.960504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:34.960504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:34.960504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:34.960504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:34.961678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 353/695] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:34.967035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.971101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:35.050831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d23fa8-0e96-4cb9-a01f-fcb572604c04 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.073081 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:35.123392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.123999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.124276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.124544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:35.125191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:35.125529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:35.127703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:35.132039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769f19-688f-46f3-b250-8c0d7f527173 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.141241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 343/696] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:35.142308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:35.174822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.175501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.175763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.176035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:35.176721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:35.177023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:35.178459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:35.196047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 354/697] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:35.196047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:35.264022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d99fa4-bd14-4af1-8fa4-9397b6c9a1f1 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.294249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:35.310706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.311625 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.311881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.312124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:35.323986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:35.325685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:35.332208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c653ad-db89-4c9b-9a9e-dbfa80b10c40 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:35.342071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:35.343032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:35.343905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:35.343905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:35.343905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:35.348409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 344/698] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:35.349889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:35.370104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:35.382710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.383976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.384725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.385604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:35.395072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:35.397232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:35.412617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:35.413541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:35.414447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:35.417540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 355/699] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:46:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:35.419393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:35.477705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21351-314d-4a66-855d-b2f5bd6877c7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.516076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.516684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.516966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.517241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:35.517951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=91df91b1ea4949119245c86bf52e32f6) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:35.518257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:35.519697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:35.528619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 345/700] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:35 2022] GET /identity/v3/role_assignments?user.id=91df91b1ea4949119245c86bf52e32f6&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:35.529603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:35.579507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:35.620720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7320f0-e193-4939-a8d0-6235729e1f01 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.697945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.698565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.698845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.699157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:35.699820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:35.700122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:35.703466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:35.724670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a42a7-f85b-4c19-87f5-e50cd8c02091 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.733264 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 356/701] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:35.738751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:35.780088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.781193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.781615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.782185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:35.783504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:35.784019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:35.785888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:35.810969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 346/702] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:46:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:35.815271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:35.817660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:35.878401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0287d4e-fff5-4163-abb1-51fb719d8c19 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.919240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8fff-5690-4dd4-bd32-1d83275d9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.920308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8fff-5690-4dd4-bd32-1d83275d9c9a None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.920775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8fff-5690-4dd4-bd32-1d83275d9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.921512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8fff-5690-4dd4-bd32-1d83275d9c9a None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:35.922379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca8fff-5690-4dd4-bd32-1d83275d9c9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:35.923990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca8fff-5690-4dd4-bd32-1d83275d9c9a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:35.928391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:35.938541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfedb3-ba78-465c-ab1e-4c9d891d92ef None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:35.951919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 357/703] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:35 2022] POST /identity/v3/users => generated 254 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:46:35.979674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651286e1-3e32-4583-ba73-3c7bd94f7130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:35.980464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651286e1-3e32-4583-ba73-3c7bd94f7130 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:35.980864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651286e1-3e32-4583-ba73-3c7bd94f7130 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:35.981199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651286e1-3e32-4583-ba73-3c7bd94f7130 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:36.058848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651286e1-3e32-4583-ba73-3c7bd94f7130 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:36.060410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651286e1-3e32-4583-ba73-3c7bd94f7130 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:36.168400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:36.179540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 347/704] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:46:36.185025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8fff-5690-4dd4-bd32-1d83275d9c9a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:36.250837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8915d6-5400-4f0e-9c76-dc079930f159 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:36.251194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8915d6-5400-4f0e-9c76-dc079930f159 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:36.251406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8915d6-5400-4f0e-9c76-dc079930f159 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:36.252955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 348/705] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:36.259079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed6b9-e746-4b51-9427-d82c25672288 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:36.259624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed6b9-e746-4b51-9427-d82c25672288 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:36.260120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed6b9-e746-4b51-9427-d82c25672288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:36.281811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-65fed6b9-e746-4b51-9427-d82c25672288 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:36.283529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-65fed6b9-e746-4b51-9427-d82c25672288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:36.298870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8fff-5690-4dd4-bd32-1d83275d9c9a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:36.301322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:36.337053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:36.337691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:36.337972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:36.338240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:36.349782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:36.351249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:36.366246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:36.367279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:36.368164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:36.368164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:36.368164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:36.368164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 358/706] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:36.373026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:36.377261 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 349/707] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:36.478132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc36e-1723-43ac-8a83-c1add1bda792 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:36.513545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:36.514155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:36.514434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:36.514701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:36.515381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:36.515682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:36.517027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:36.529472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 359/708] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:36.534462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fed6b9-e746-4b51-9427-d82c25672288 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:36.670432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fed6b9-e746-4b51-9427-d82c25672288 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:36.700527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:36.717987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:36.718625 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:36.718916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:36.719195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:36.732428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:36.733809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:36.748449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:36.749372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:36.750921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 350/709] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:36.755193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:36.786583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:36.849595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:36.893196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52519cc5-4f74-4685-9649-31727f77b5dc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:36.933791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621190e-93ae-4e6c-be95-c977e607da71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:36.934400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621190e-93ae-4e6c-be95-c977e607da71 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:36.934680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621190e-93ae-4e6c-be95-c977e607da71 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:36.934979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621190e-93ae-4e6c-be95-c977e607da71 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:36.935681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621190e-93ae-4e6c-be95-c977e607da71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:36.935982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621190e-93ae-4e6c-be95-c977e607da71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:36.937401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621190e-93ae-4e6c-be95-c977e607da71 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:36.952305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 360/710] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:36.953683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8895-705a-41f1-9849-4ed85f7bb18f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:36.953979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8895-705a-41f1-9849-4ed85f7bb18f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:36.954250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8895-705a-41f1-9849-4ed85f7bb18f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:36.960097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 361/711] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:36.960097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8895-705a-41f1-9849-4ed85f7bb18f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:36.971770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a888c1f-9493-4302-8a57-bf4ec0c33f15 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:37.007695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cd00e-c0d3-4412-8abd-24857944adb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:37.008306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cd00e-c0d3-4412-8abd-24857944adb9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.008585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cd00e-c0d3-4412-8abd-24857944adb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.008885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cd00e-c0d3-4412-8abd-24857944adb9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.009468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cd00e-c0d3-4412-8abd-24857944adb9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:37.010827 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cd00e-c0d3-4412-8abd-24857944adb9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-1d1cd00e-c0d3-4412-8abd-24857944adb9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:37.021967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:37.023014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:37.024432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 351/712] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Oct 10 11:46:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:37.025200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3a34-db66-48c9-99e3-25a47e155175 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.025200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3a34-db66-48c9-99e3-25a47e155175 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.025200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3a34-db66-48c9-99e3-25a47e155175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.043805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-adac3a34-db66-48c9-99e3-25a47e155175 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:37.044982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-adac3a34-db66-48c9-99e3-25a47e155175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:37.054052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:37.089316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8895-705a-41f1-9849-4ed85f7bb18f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:37.091777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:37.124388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 352/713] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:37.125226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac3a34-db66-48c9-99e3-25a47e155175 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:37.138788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:37.139910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.140808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.141260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.142423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=152edb19fd7b434485b0e1ebc5078335) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:37.142874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:37.145098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:37.152989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 362/714] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:36 2022] GET /identity/v3/role_assignments?user.id=152edb19fd7b434485b0e1ebc5078335&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:37.242306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac3a34-db66-48c9-99e3-25a47e155175 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:37.293988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:37.294540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.294880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.295175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.295816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:37.296119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:37.297487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:37.302817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 353/715] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Oct 10 11:46:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:37.308337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:37.429171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:37.435247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef157e-e0b7-430f-9327-4214323fdea0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:37.480443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:37.482084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:37.482722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.484058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.484341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.485019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:37.485325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:37.487563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:37.503645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 363/716] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Oct 10 11:46:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:37.576788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde4f02-8599-4a90-85d3-fc95c178ceee None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:37.628833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:37.629487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.629763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.630030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] PATH_INFO: `/v3/services/network` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.631206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:37.632568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:37.648192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:37.649214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:37.650024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 354/717] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:37 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:37.655385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:37.663783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:37.810929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82541096-96bc-4e42-a515-21a31d6b011c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:37.848673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6a2a8-03e9-41e3-976b-c02611b2c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:37.849231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6a2a8-03e9-41e3-976b-c02611b2c073 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.849508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6a2a8-03e9-41e3-976b-c02611b2c073 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.849779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6a2a8-03e9-41e3-976b-c02611b2c073 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.850468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6a2a8-03e9-41e3-976b-c02611b2c073 None admin] RBAC: Adding query filter params (name=network) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:37.850773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6a2a8-03e9-41e3-976b-c02611b2c073 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:37.852169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6a2a8-03e9-41e3-976b-c02611b2c073 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:37.857039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 364/718] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:46:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:37.862227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:38.000014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1680b-d22d-4d14-be38-87756a318fcd None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:38.007923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:38.053464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416cf7d-7607-48c4-8061-49be01f7ef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:38.054075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cf7d-7607-48c4-8061-49be01f7ef05 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.054353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cf7d-7607-48c4-8061-49be01f7ef05 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.054621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cf7d-7607-48c4-8061-49be01f7ef05 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.055274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416cf7d-7607-48c4-8061-49be01f7ef05 None admin] RBAC: Adding query filter params (type=network) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:38.055570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416cf7d-7607-48c4-8061-49be01f7ef05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:38.056935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416cf7d-7607-48c4-8061-49be01f7ef05 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:38.068426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 355/719] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:46:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:38.838456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911f5d-4b90-45fa-8a6d-81d043a0c640 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.839493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911f5d-4b90-45fa-8a6d-81d043a0c640 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.840004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911f5d-4b90-45fa-8a6d-81d043a0c640 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.841818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 365/720] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:38.847326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72adb-b1e8-4942-afd4-a9a04e303811 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.847842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72adb-b1e8-4942-afd4-a9a04e303811 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.848331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72adb-b1e8-4942-afd4-a9a04e303811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.852889 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0c4ed-f607-486a-ba83-cdd24f353683 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.853406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0c4ed-f607-486a-ba83-cdd24f353683 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.853880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0c4ed-f607-486a-ba83-cdd24f353683 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.855572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 366/721] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:38.864066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c9c59-5af3-43c3-8323-e90508738d58 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.864350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c9c59-5af3-43c3-8323-e90508738d58 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.865160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c9c59-5af3-43c3-8323-e90508738d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.875731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-43f72adb-b1e8-4942-afd4-a9a04e303811 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:38.877504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-43f72adb-b1e8-4942-afd4-a9a04e303811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:38.896256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d91c9c59-5af3-43c3-8323-e90508738d58 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:38.897858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d91c9c59-5af3-43c3-8323-e90508738d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:38.973541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:38.974203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 367/722] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:39.004450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 356/723] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:39.287740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c9c59-5af3-43c3-8323-e90508738d58 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.389775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c9c59-5af3-43c3-8323-e90508738d58 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.452510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aae81a-82b4-4b27-b10e-31eccfb57be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.453157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aae81a-82b4-4b27-b10e-31eccfb57be4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.453435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aae81a-82b4-4b27-b10e-31eccfb57be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.453710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aae81a-82b4-4b27-b10e-31eccfb57be4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.471755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aae81a-82b4-4b27-b10e-31eccfb57be4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-10aae81a-82b4-4b27-b10e-31eccfb57be4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:39.477058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:39.478022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:39.478845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 368/724] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:39.486025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72adb-b1e8-4942-afd4-a9a04e303811 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.493892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aae81a-82b4-4b27-b10e-31eccfb57be4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.607811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72adb-b1e8-4942-afd4-a9a04e303811 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.611231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.647535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20928cb2-9b20-4f78-9476-014ce0ea6333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.648507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20928cb2-9b20-4f78-9476-014ce0ea6333 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.648981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20928cb2-9b20-4f78-9476-014ce0ea6333 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.649465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20928cb2-9b20-4f78-9476-014ce0ea6333 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.651035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20928cb2-9b20-4f78-9476-014ce0ea6333 None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:39.651348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20928cb2-9b20-4f78-9476-014ce0ea6333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.652767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20928cb2-9b20-4f78-9476-014ce0ea6333 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.658397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.666851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 357/725] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:39.666851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab21434-b53d-41ad-85df-12f2a1b195da None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.666851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab21434-b53d-41ad-85df-12f2a1b195da None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.666851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab21434-b53d-41ad-85df-12f2a1b195da None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.666851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 358/726] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:39.667766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab21434-b53d-41ad-85df-12f2a1b195da None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.717024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aae81a-82b4-4b27-b10e-31eccfb57be4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.760161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431f19d-a073-43c1-bb54-76d6563eec80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.761510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431f19d-a073-43c1-bb54-76d6563eec80 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.761994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431f19d-a073-43c1-bb54-76d6563eec80 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.762726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431f19d-a073-43c1-bb54-76d6563eec80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.774744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab21434-b53d-41ad-85df-12f2a1b195da None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.776427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431f19d-a073-43c1-bb54-76d6563eec80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5431f19d-a073-43c1-bb54-76d6563eec80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:39.782567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:39.783728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:39.784726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:39.788205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 369/727] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:39.789909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73e741-846a-4f1c-950b-1dc6c9d430d5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.790335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73e741-846a-4f1c-950b-1dc6c9d430d5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.791293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73e741-846a-4f1c-950b-1dc6c9d430d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.815318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3a73e741-846a-4f1c-950b-1dc6c9d430d5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:39.816616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3a73e741-846a-4f1c-950b-1dc6c9d430d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:39.818768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.819769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.820958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.821381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.833325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.835618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:39.853483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:39.854622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:39.855731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:39.855731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:39.855731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:39.862797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 359/728] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:39.862797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.881828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.900846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 370/729] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:39.901819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73e741-846a-4f1c-950b-1dc6c9d430d5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.954792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.990478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af14c-9ed4-4792-b24a-2e00bcaf8190 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.012390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73e741-846a-4f1c-950b-1dc6c9d430d5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.036027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.037169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.037441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.037706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.038360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:40.038654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.040054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.048795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 360/730] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:40.054273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.055057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.055404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.055707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.055812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.056536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:40.056893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.058516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.077095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 371/731] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:40.082744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.150617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.207495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd3a92-3336-41c7-83d4-ef670cde4aef None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.212962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf5ef-a926-4eec-8069-1ca7006a45cc None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.223230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.269800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.271215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.271495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.271768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.279527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.280158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.280429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.280692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.286037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.287659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.297819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.299289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:40.317257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:40.318866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 372/732] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:40.320227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:40.326710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:40.327680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:40.329866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:40.329866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:40.329866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:40.329866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 361/733] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:40.329866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.446848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640db256-1eb9-4ca3-9c49-60835329671c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.463305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ec740-e979-40e1-b1d0-4ccaebb5cda5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.487195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6bea-de29-4f49-bc4f-667d12ed8941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.487819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6bea-de29-4f49-bc4f-667d12ed8941 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.488101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6bea-de29-4f49-bc4f-667d12ed8941 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.488371 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6bea-de29-4f49-bc4f-667d12ed8941 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.492293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.499452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a6bea-de29-4f49-bc4f-667d12ed8941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:40.502459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1efb81-e0fc-4357-91bd-9fbcf2a96092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.503326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1efb81-e0fc-4357-91bd-9fbcf2a96092 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.503833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1efb81-e0fc-4357-91bd-9fbcf2a96092 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.504304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1efb81-e0fc-4357-91bd-9fbcf2a96092 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.505787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1efb81-e0fc-4357-91bd-9fbcf2a96092 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4c1a6bea-de29-4f49-bc4f-667d12ed8941 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:40.505894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:40.506859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 373/734] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:40.507708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6bea-de29-4f49-bc4f-667d12ed8941 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.509437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1efb81-e0fc-4357-91bd-9fbcf2a96092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.510872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1efb81-e0fc-4357-91bd-9fbcf2a96092 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.528786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 362/735] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:40.528786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179183-432c-4aa0-89e7-485306a15f62 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.528786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179183-432c-4aa0-89e7-485306a15f62 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.528786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179183-432c-4aa0-89e7-485306a15f62 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.532041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 363/736] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:40.532676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb179183-432c-4aa0-89e7-485306a15f62 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.543427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.622132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6bea-de29-4f49-bc4f-667d12ed8941 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.648738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb179183-432c-4aa0-89e7-485306a15f62 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.655667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.656207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.656485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.656750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.657377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:40.657670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.659045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.672599 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 374/737] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:40.673284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.687152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c043bf9-30fa-4166-bbd8-24ef51d006d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.687759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c043bf9-30fa-4166-bbd8-24ef51d006d3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.688292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c043bf9-30fa-4166-bbd8-24ef51d006d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.688559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c043bf9-30fa-4166-bbd8-24ef51d006d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.689209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c043bf9-30fa-4166-bbd8-24ef51d006d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:40.689508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c043bf9-30fa-4166-bbd8-24ef51d006d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.690870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c043bf9-30fa-4166-bbd8-24ef51d006d3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.698931 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 364/738] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:40.699978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd4057-9038-4a89-ac1e-1503184993ce None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.700303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd4057-9038-4a89-ac1e-1503184993ce None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.700542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd4057-9038-4a89-ac1e-1503184993ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.717330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-76fd4057-9038-4a89-ac1e-1503184993ce None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:40.718504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-76fd4057-9038-4a89-ac1e-1503184993ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:40.759837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.761135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.775716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e1225-641e-4f25-a20d-2705bdda6de8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.785919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 365/739] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:40.787780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985625b-7019-4040-a431-0ce4d1476b5e None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.788070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985625b-7019-4040-a431-0ce4d1476b5e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.788339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985625b-7019-4040-a431-0ce4d1476b5e None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.789556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 366/740] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:40.790241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985625b-7019-4040-a431-0ce4d1476b5e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.827747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f84aae-3aba-4116-a43e-d87a8e82ae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.828361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f84aae-3aba-4116-a43e-d87a8e82ae40 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.828641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f84aae-3aba-4116-a43e-d87a8e82ae40 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.828911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f84aae-3aba-4116-a43e-d87a8e82ae40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.899245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985625b-7019-4040-a431-0ce4d1476b5e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.900826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f84aae-3aba-4116-a43e-d87a8e82ae40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.902590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f84aae-3aba-4116-a43e-d87a8e82ae40 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.934401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.934984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.935269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.935542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] PATH_INFO: `/v3/users/swift` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.942932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.944363 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:40.955588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:40.956503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 367/741] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:40.957416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.999547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.002161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.013412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 375/742] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:41.017988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7a85a-7e8a-4db8-96de-27478f80f9cc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.017988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7a85a-7e8a-4db8-96de-27478f80f9cc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.017988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7a85a-7e8a-4db8-96de-27478f80f9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.033893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-efc7a85a-7e8a-4db8-96de-27478f80f9cc None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:41.035109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-efc7a85a-7e8a-4db8-96de-27478f80f9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:41.072431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7937a8a-27b4-40be-870d-f7307fff30df None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.102429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 376/743] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:41.103277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7a85a-7e8a-4db8-96de-27478f80f9cc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.118863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.119524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.119806 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.120256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.137643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.139106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:41.156281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:41.157238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:41.158313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:41.158313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:41.158313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:41.158313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 368/744] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:41.162947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.203085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7a85a-7e8a-4db8-96de-27478f80f9cc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.220133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.238483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.239103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.239390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.239691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.240415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:41.240711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.242108 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.263061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 377/745] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:41.263061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.267061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.280942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000576d-f034-4b95-a7f4-583f6bb5f765 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.316004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.316302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.316585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.316916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.327552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.328969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:41.343654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:41.344672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:41.345550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:41.345550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:41.345550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:41.345550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 369/746] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:41.345550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.371150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8bbbe-c98d-40d9-9a17-12b8897f0ead None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.407517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733b535-b835-4ada-810f-a6366e50cb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.408124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b535-b835-4ada-810f-a6366e50cb11 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.408404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b535-b835-4ada-810f-a6366e50cb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.408675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b535-b835-4ada-810f-a6366e50cb11 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.409322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733b535-b835-4ada-810f-a6366e50cb11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.410693 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733b535-b835-4ada-810f-a6366e50cb11 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.428567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 378/747] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:46:41 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:41.429387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733b535-b835-4ada-810f-a6366e50cb11 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.456596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28995552-2aa7-4e13-a443-236ec5ba3522 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.469878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.502808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.503749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.504199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.504609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.505961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:41.506263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.507697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.517082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.528497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 370/748] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:41.529468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.549595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733b535-b835-4ada-810f-a6366e50cb11 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.590360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936ab0c-694d-435b-a4a3-1f257ca114a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.591018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936ab0c-694d-435b-a4a3-1f257ca114a1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.591296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936ab0c-694d-435b-a4a3-1f257ca114a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.591616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936ab0c-694d-435b-a4a3-1f257ca114a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.643391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d372a-d859-4c46-988c-7a6251777209 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.671394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936ab0c-694d-435b-a4a3-1f257ca114a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.672832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936ab0c-694d-435b-a4a3-1f257ca114a1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.680658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.681481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.681937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.682509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.683359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:41.683897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.685407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.703126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 371/749] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:41.704052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.707193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.764927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.777872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 379/750] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:41.778718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936ab0c-694d-435b-a4a3-1f257ca114a1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.816972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2be6dc-0ec8-4eca-93d9-075caa2a2781 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.854048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20544e6-6725-4e62-8a0d-99b26cce1833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.854940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20544e6-6725-4e62-8a0d-99b26cce1833 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.855359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20544e6-6725-4e62-8a0d-99b26cce1833 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.855877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20544e6-6725-4e62-8a0d-99b26cce1833 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.856521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20544e6-6725-4e62-8a0d-99b26cce1833 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.857979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20544e6-6725-4e62-8a0d-99b26cce1833 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.876453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 372/751] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:46:41 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:41.877284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20544e6-6725-4e62-8a0d-99b26cce1833 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.894950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936ab0c-694d-435b-a4a3-1f257ca114a1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.944644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.950715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.951792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.952368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.952711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.961481 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.963093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:41.975912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:41.976907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:41.977850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:41.982234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 380/752] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:41.983259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.031693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20544e6-6725-4e62-8a0d-99b26cce1833 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.057591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:42.089684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.090431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.090711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.091040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.099855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.101303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.112066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014a239-a07f-4605-9041-c4bed1cd68fc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:42.113403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:42.114300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 373/753] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:42.115328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.161753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.163323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.163821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.164297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.180292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.182948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:42.205712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:42.206918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:42.208026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:42.208026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:42.208026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:42.210082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 381/754] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:42.211396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.231102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98aae4-b859-440f-a9d7-8470b77dd278 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.266447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.267248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.267621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.268037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.268948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:42.269648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.271459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.277565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:42.295224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 374/755] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:42.296478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.337755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:42.352667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0788b4f-de70-4dd1-b133-999621c42286 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.406397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.407562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.408411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.408998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.409955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:42.410661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.413085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.420525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98985add-41c3-4859-a173-b7723cbb7b1f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.434140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 382/756] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:42.434140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.462841 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.463516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.463795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.464061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.464694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:42.464989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.466330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.478163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 375/757] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:42.479091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.568483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7501e-addb-46f8-b75d-52a5335d31e6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.571439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:42.587083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:42.596231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bac02c-dcdb-4228-936c-8c4112b760f4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.606487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.606809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.607071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.607329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.618905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.620362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:42.636400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:42.637379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:42.638265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:42.638265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:42.638265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:42.642251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 383/758] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:42.643162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.644095 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.644937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.645417 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.646128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.647578 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=cf591850b9914f2f9089c8b6df730b4a) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:42.647694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.650673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.662936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 376/759] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/role_assignments?user.id=cf591850b9914f2f9089c8b6df730b4a&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:42.662936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.852577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69e2a7-538b-4c12-8266-2c1f5e31dbea None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.854543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f453b-16ec-4d55-8135-6aa9c6c036f0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:42.910366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.911040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.911853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.912228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.919465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.921082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.924155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:42.924781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.925067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.925342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.925989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:42.926292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:42.928225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:42.933034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:42.933900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:42.934797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 384/760] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:42.940782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.954136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 377/761] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:42.961286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:42.965754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:43.016219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:43.068535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c9850-49ef-4c3c-abe4-7057eeb14060 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.106280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:43.106812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:43.107118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:43.107386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:43.108077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:43.108378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:43.109790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:43.123793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 385/762] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:43.131884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:43.148177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5546b5-e1e1-464c-ad8e-e4f25552ec07 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.205009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:43.205654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:43.205936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:43.206211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:43.216499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:43.217964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:43.236422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:43.237362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:43.238240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 378/763] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:43.247855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:43.275692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:43.280304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abecb5-a8a2-4868-bb81-0dd64e4daf5d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.349550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:43.350535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:43.351071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:43.351348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:43.352095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=91df91b1ea4949119245c86bf52e32f6) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:43.352396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:43.353908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:43.368045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 386/764] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:43 2022] GET /identity/v3/role_assignments?user.id=91df91b1ea4949119245c86bf52e32f6&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:43.383289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:43.412533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203757fe-df3d-4933-8019-eee2d788f1cf None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.452223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82300c4f-4c93-46b8-8989-e2427869fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:43.452853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82300c4f-4c93-46b8-8989-e2427869fe52 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:43.453134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82300c4f-4c93-46b8-8989-e2427869fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:43.453406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82300c4f-4c93-46b8-8989-e2427869fe52 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:43.454117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82300c4f-4c93-46b8-8989-e2427869fe52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:43.454423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82300c4f-4c93-46b8-8989-e2427869fe52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:43.455956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82300c4f-4c93-46b8-8989-e2427869fe52 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:43.478747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 379/765] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:43.484215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:43.615055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763d40c-4cf7-4ab0-aebd-4b61d565c598 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.651389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4201a2b-1433-466f-9e4f-86db75951a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:43.651961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4201a2b-1433-466f-9e4f-86db75951a35 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:43.652249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4201a2b-1433-466f-9e4f-86db75951a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:43.652523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4201a2b-1433-466f-9e4f-86db75951a35 None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/152edb19fd7b434485b0e1ebc5078335/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:43.654544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4201a2b-1433-466f-9e4f-86db75951a35 None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=152edb19fd7b434485b0e1ebc5078335, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:43.656053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4201a2b-1433-466f-9e4f-86db75951a35 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:43.684751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:43.690432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 387/766] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:43 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/152edb19fd7b434485b0e1ebc5078335/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:44.089024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d513-9e64-4dfc-90f2-cd015ea9589e None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.089420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d513-9e64-4dfc-90f2-cd015ea9589e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.089599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d513-9e64-4dfc-90f2-cd015ea9589e None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.091109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 380/767] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:44.096343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644038e-559f-4fdf-90cf-3230e95f351f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.097215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644038e-559f-4fdf-90cf-3230e95f351f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.097759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644038e-559f-4fdf-90cf-3230e95f351f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.123931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2644038e-559f-4fdf-90cf-3230e95f351f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:44.125145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2644038e-559f-4fdf-90cf-3230e95f351f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:44.209324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 388/768] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:44.362773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7281d2-7d9a-442e-848e-b3cd51b38c42 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.365891 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7281d2-7d9a-442e-848e-b3cd51b38c42 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.365891 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7281d2-7d9a-442e-848e-b3cd51b38c42 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.366378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 381/769] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:44.371238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd328fa-4e6a-424b-9edf-eb3b3de10952 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.371535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd328fa-4e6a-424b-9edf-eb3b3de10952 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.371814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd328fa-4e6a-424b-9edf-eb3b3de10952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.402168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3dd328fa-4e6a-424b-9edf-eb3b3de10952 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:44.402168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3dd328fa-4e6a-424b-9edf-eb3b3de10952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:44.472915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 389/770] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:44.620125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7281d2-7d9a-442e-848e-b3cd51b38c42 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:44.817221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd328fa-4e6a-424b-9edf-eb3b3de10952 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:44.887722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7281d2-7d9a-442e-848e-b3cd51b38c42 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:44.903289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:44.947534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:44.948172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.948452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.948723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.949309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:44.950207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd328fa-4e6a-424b-9edf-eb3b3de10952 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:44.950670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:44.972263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:44.973284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:44.974693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 382/771] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:44.979844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:44.982756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:44.992264 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:44.993190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.994259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.994736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.995550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:44.997104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:45.014121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:45.015209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:45.016039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:45.026277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 390/772] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:45.031035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.129030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b779-887e-4045-8c95-a61883fdb1d3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.171919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27395192-d6aa-4cc5-9366-beba407716f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.172870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27395192-d6aa-4cc5-9366-beba407716f8 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.173356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27395192-d6aa-4cc5-9366-beba407716f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.173989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27395192-d6aa-4cc5-9366-beba407716f8 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.175343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27395192-d6aa-4cc5-9366-beba407716f8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:45.175878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27395192-d6aa-4cc5-9366-beba407716f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.178479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27395192-d6aa-4cc5-9366-beba407716f8 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.184990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 383/773] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.186844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69067d2-f005-4287-bb53-d5f6fbcf384d None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.187360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69067d2-f005-4287-bb53-d5f6fbcf384d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.188447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69067d2-f005-4287-bb53-d5f6fbcf384d None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.190208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 384/774] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:45.191246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69067d2-f005-4287-bb53-d5f6fbcf384d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.210160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593db0f-67cb-45c8-a78a-36503b3248e6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.239133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.247619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd672-0f75-416c-9265-38125cc14840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.248490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd672-0f75-416c-9265-38125cc14840 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.249057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd672-0f75-416c-9265-38125cc14840 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.249532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd672-0f75-416c-9265-38125cc14840 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.250436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacd672-0f75-416c-9265-38125cc14840 None admin] RBAC: Adding query filter params (name=network) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:45.250726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacd672-0f75-416c-9265-38125cc14840 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.252307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacd672-0f75-416c-9265-38125cc14840 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.259754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 391/775] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:45.261179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a02a6-8444-415f-ae42-a8da826b75fd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.261610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a02a6-8444-415f-ae42-a8da826b75fd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.262029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a02a6-8444-415f-ae42-a8da826b75fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.283327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.283554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d69a02a6-8444-415f-ae42-a8da826b75fd None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:45.285148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d69a02a6-8444-415f-ae42-a8da826b75fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:45.321505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69067d2-f005-4287-bb53-d5f6fbcf384d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.359418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.360561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.361091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.361561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.362521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:45.363151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.365809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.369104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 392/776] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:45.370433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a02a6-8444-415f-ae42-a8da826b75fd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.376376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 385/777] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.377323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.504411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a02a6-8444-415f-ae42-a8da826b75fd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.507869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c457f6-e4ba-4dd4-a26f-5af1ec501416 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.526976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.548237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.555564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.556365 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.556645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.556913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.557608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] RBAC: Adding query filter params (interface=public, service_id=eb55007c953b4028b86df256ad8782bd, region_id=RegionOne) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:45.557914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.559293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.568022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.568631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.568908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.569176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.569804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] RBAC: Adding query filter params (type=network) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:45.570101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.571516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.575816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 386/778] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/endpoints?service_id=eb55007c953b4028b86df256ad8782bd&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.579986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 393/779] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.585828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.588230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.751037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354b9a6-587a-40ef-a09a-3d962c62e68f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.754292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7f66b-bdc9-4672-ae90-6d8181ba9800 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.788358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d41040-ee4f-472e-a8ba-af42245a194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.789362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41040-ee4f-472e-a8ba-af42245a194c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.789844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41040-ee4f-472e-a8ba-af42245a194c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.790304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41040-ee4f-472e-a8ba-af42245a194c None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.791136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d41040-ee4f-472e-a8ba-af42245a194c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.792997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d41040-ee4f-472e-a8ba-af42245a194c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.794836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.795478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.795758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.796031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.796731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] RBAC: Adding query filter params (interface=public, service_id=9fb871089653474685131e6e549889ea, region_id=RegionOne) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:45.797043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.798383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.803984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 387/780] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/endpoints?service_id=9fb871089653474685131e6e549889ea&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.811096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.811266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 394/781] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.815729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d41040-ee4f-472e-a8ba-af42245a194c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.916227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.940165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.017497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbd9fc-d5f8-41af-b204-cf24013e7094 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.018205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d41040-ee4f-472e-a8ba-af42245a194c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.062541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b75801-b7a4-48e4-8fe7-ef1c4e20f6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.064057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75801-b7a4-48e4-8fe7-ef1c4e20f6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.064637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75801-b7a4-48e4-8fe7-ef1c4e20f6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.065103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b75801-b7a4-48e4-8fe7-ef1c4e20f6c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.065216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a99e9-9199-4bd9-8b92-e1c0200b0475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.065723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a99e9-9199-4bd9-8b92-e1c0200b0475 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.066007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a99e9-9199-4bd9-8b92-e1c0200b0475 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.066290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a99e9-9199-4bd9-8b92-e1c0200b0475 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.066885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a99e9-9199-4bd9-8b92-e1c0200b0475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.069253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a99e9-9199-4bd9-8b92-e1c0200b0475 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.078924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 395/782] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/services => generated 772 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:46.082180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b75801-b7a4-48e4-8fe7-ef1c4e20f6c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-87b75801-b7a4-48e4-8fe7-ef1c4e20f6c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:46.087904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:46.088898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:46.089731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 388/783] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:46.095330 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a99e9-9199-4bd9-8b92-e1c0200b0475 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.228094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a99e9-9199-4bd9-8b92-e1c0200b0475 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.240437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.272395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a686f-4f93-46f3-8150-b7b5226f6ed6 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.272815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a686f-4f93-46f3-8150-b7b5226f6ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.272986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a686f-4f93-46f3-8150-b7b5226f6ed6 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.274423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6fb50-a87f-451c-aeef-aef9249aeda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.275075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6fb50-a87f-451c-aeef-aef9249aeda0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.275353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6fb50-a87f-451c-aeef-aef9249aeda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.275625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6fb50-a87f-451c-aeef-aef9249aeda0 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.276303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6fb50-a87f-451c-aeef-aef9249aeda0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:46.276499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 389/784] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:46.276685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6fb50-a87f-451c-aeef-aef9249aeda0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.278088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6fb50-a87f-451c-aeef-aef9249aeda0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.280174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4405a-5d8b-40ca-aa13-2022d1bbb84c None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.280460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4405a-5d8b-40ca-aa13-2022d1bbb84c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.280738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4405a-5d8b-40ca-aa13-2022d1bbb84c None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.282349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 390/785] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:46.283845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552f8a-875f-414a-a415-f96efea22c26 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.284136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552f8a-875f-414a-a415-f96efea22c26 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.284406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552f8a-875f-414a-a415-f96efea22c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.288050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 396/786] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:46.289303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578298a0-ca83-4372-8f4f-a313be1a0051 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.289601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578298a0-ca83-4372-8f4f-a313be1a0051 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.289871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578298a0-ca83-4372-8f4f-a313be1a0051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.308860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-2a552f8a-875f-414a-a415-f96efea22c26 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:46.310086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2a552f8a-875f-414a-a415-f96efea22c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:46.311852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-578298a0-ca83-4372-8f4f-a313be1a0051 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:46.314795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-578298a0-ca83-4372-8f4f-a313be1a0051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:46.394669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 391/787] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:46:46.395529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a552f8a-875f-414a-a415-f96efea22c26 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.419909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 397/788] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:46.468633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.535284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a552f8a-875f-414a-a415-f96efea22c26 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.583763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.584403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.584675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.584938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.600136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.601564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:46.618559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:46.619581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:46.620460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:46.620460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.620460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:46.620460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 392/789] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:46.625505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578298a0-ca83-4372-8f4f-a313be1a0051 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.766389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578298a0-ca83-4372-8f4f-a313be1a0051 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.768381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.775865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.809347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.809965 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.810245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.810516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.812064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:46.812064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.813498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.827981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 398/790] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:46.828840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.912226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a5ac-b71f-46ee-8bd6-da1d48149147 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.915481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.947993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad35d27-7fb6-4617-9174-a62da5a72808 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.961188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149f4cc-9faa-4c7c-8e53-2892cccf833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.962296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149f4cc-9faa-4c7c-8e53-2892cccf833d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.963225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149f4cc-9faa-4c7c-8e53-2892cccf833d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.963833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149f4cc-9faa-4c7c-8e53-2892cccf833d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.978412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149f4cc-9faa-4c7c-8e53-2892cccf833d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:46.983757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968b34f-9943-41c7-9c04-3d320dc85f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.984468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b34f-9943-41c7-9c04-3d320dc85f85 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.984875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b34f-9943-41c7-9c04-3d320dc85f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.985221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b34f-9943-41c7-9c04-3d320dc85f85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-f149f4cc-9faa-4c7c-8e53-2892cccf833d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:46.985888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:46.986780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:46.987689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:46.988393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 393/791] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:46.993408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149f4cc-9faa-4c7c-8e53-2892cccf833d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.006020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968b34f-9943-41c7-9c04-3d320dc85f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2968b34f-9943-41c7-9c04-3d320dc85f85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:47.012565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:47.013580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:47.014435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:47.017174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 399/792] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:47.018445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968b34f-9943-41c7-9c04-3d320dc85f85 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.095271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.108862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149f4cc-9faa-4c7c-8e53-2892cccf833d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.143383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968b34f-9943-41c7-9c04-3d320dc85f85 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.146476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.147505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.148052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.148527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.157227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.159040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:47.172344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:47.173281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:46:47.174236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:47.176776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 394/793] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:47.177847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.190935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.190935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.190935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.190935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.191297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.191297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.192287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.201654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.209558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 400/794] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.211477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.307334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb777f3c-0cdd-4497-8112-c414dd6a5f78 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.310881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.329936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19ba5d-ba67-4c77-8fd4-07113f088980 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.357987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.358904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.359320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.359678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.360415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.360787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.362226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.374271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.375180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.375696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.376128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 395/795] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.376360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.377141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.377590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.378129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.380729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.405737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 401/796] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.407189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.471947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.500105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62217e19-fb0d-42ae-8e96-039e1eebd068 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.536548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.537465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c854b09-0c41-4b3c-aeaa-f02bfee020ef None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.537756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.538253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.538716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.549740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.551462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:47.566263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:47.567819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:47.568834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:47.568834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:47.568834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:47.575036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 396/797] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:47.576278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.578488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789195d7-1388-483f-b351-6096504a6464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.579165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789195d7-1388-483f-b351-6096504a6464 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.579488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789195d7-1388-483f-b351-6096504a6464 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.579766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789195d7-1388-483f-b351-6096504a6464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.593996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789195d7-1388-483f-b351-6096504a6464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.595644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789195d7-1388-483f-b351-6096504a6464 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.597587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-789195d7-1388-483f-b351-6096504a6464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:47.616394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:47.617334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:47.618207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:47.618207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:47.618207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:47.618207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 402/798] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:47.618721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789195d7-1388-483f-b351-6096504a6464 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.688856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f063d7-d3f1-4c7d-8917-11f8c7d5b08a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.728498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.729379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.729820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.730230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.732639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.745012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.746699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:47.762301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:47.763594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:47.764636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:47.764636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:47.764636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:47.766370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 397/799] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:47.767322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.873160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21c014-8bae-4c70-b652-cce8a66a02de None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.878878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789195d7-1388-483f-b351-6096504a6464 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.908115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.908985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.909414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.909835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.910644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.911166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.912654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.920465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.921623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.921905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.922175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.922819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.923165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.925400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.929579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 398/800] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.930681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.943976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 403/801] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.943976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.993730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.009864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.061706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8efb3-de21-456c-b024-b66caf7dd8e5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.061938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6f4f5-27c4-4b15-8e1d-4a0c4931ee6c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.106492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.107158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.107430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.107701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.109888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.110770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.111264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.111744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.112569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:48.113077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.115373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.116084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.117945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.128158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 399/802] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:48.129250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:48.137325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:48.138236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:48.139187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:48.139734 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 404/803] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:48.140541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.245427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c0566-d911-4106-a43d-4ab617d3f5b2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.254194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.285088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f47f1-dab3-4ccf-8f71-54fab7319d11 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.292468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.293108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.293385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.293654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.303631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.304914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.306672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:48.323939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:48.324929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:48.325873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 400/804] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:48.326442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.341038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.342398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.343064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.343530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.352268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.353919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:48.366326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:48.367338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:48.368215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 405/805] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:48.368692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.451465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af7965-9dc0-4134-a469-91d90b60a331 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.483770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc5fef-e831-4fbb-8861-b57c28f90425 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.486025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61483fe3-8249-4cea-8e2f-8cc36952efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.486815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61483fe3-8249-4cea-8e2f-8cc36952efb1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.487139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61483fe3-8249-4cea-8e2f-8cc36952efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.487408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61483fe3-8249-4cea-8e2f-8cc36952efb1 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.488138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61483fe3-8249-4cea-8e2f-8cc36952efb1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:48.488443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61483fe3-8249-4cea-8e2f-8cc36952efb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.490455 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61483fe3-8249-4cea-8e2f-8cc36952efb1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.507275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 401/806] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:48.508449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af760b5b-87bf-4999-a8b8-7af1a863be15 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.508790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af760b5b-87bf-4999-a8b8-7af1a863be15 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.509071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af760b5b-87bf-4999-a8b8-7af1a863be15 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.510406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 402/807] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:48.511128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af760b5b-87bf-4999-a8b8-7af1a863be15 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.517604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17d397-398e-4398-b17c-8bd2f00b7d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.518206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17d397-398e-4398-b17c-8bd2f00b7d6e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.518490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17d397-398e-4398-b17c-8bd2f00b7d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.518764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17d397-398e-4398-b17c-8bd2f00b7d6e None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.519525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17d397-398e-4398-b17c-8bd2f00b7d6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:48.519837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17d397-398e-4398-b17c-8bd2f00b7d6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.521783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17d397-398e-4398-b17c-8bd2f00b7d6e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.537054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.539458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 406/808] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:48.540784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947ab1-8ae1-4334-a3f5-cde4cbdfbd9d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.541068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947ab1-8ae1-4334-a3f5-cde4cbdfbd9d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.541377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947ab1-8ae1-4334-a3f5-cde4cbdfbd9d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.542519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 407/809] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:48.543229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7947ab1-8ae1-4334-a3f5-cde4cbdfbd9d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.575011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.616250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af760b5b-87bf-4999-a8b8-7af1a863be15 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.645213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7947ab1-8ae1-4334-a3f5-cde4cbdfbd9d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.649229 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d53b55-fb7d-4a08-8912-7ffcba7a91d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.649826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53b55-fb7d-4a08-8912-7ffcba7a91d3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.650102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53b55-fb7d-4a08-8912-7ffcba7a91d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.650368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53b55-fb7d-4a08-8912-7ffcba7a91d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.651097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d53b55-fb7d-4a08-8912-7ffcba7a91d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:48.651374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d53b55-fb7d-4a08-8912-7ffcba7a91d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.652762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d53b55-fb7d-4a08-8912-7ffcba7a91d3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.664459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 403/810] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:48.665689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343c31-6619-4d1c-8436-4a5ea8049a63 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.665977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343c31-6619-4d1c-8436-4a5ea8049a63 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.666247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343c31-6619-4d1c-8436-4a5ea8049a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.683083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.683432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-df343c31-6619-4d1c-8436-4a5ea8049a63 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:48.683700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.683978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.684245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.684627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-df343c31-6619-4d1c-8436-4a5ea8049a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:48.694835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.696266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:48.710835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:48.711779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:48.712637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:48.712637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:48.712637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:48.712637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 408/811] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:48.712865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.752989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 404/812] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:48.754222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b9af7-997e-487f-9c40-b07efc0fb171 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.754508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b9af7-997e-487f-9c40-b07efc0fb171 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.754782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b9af7-997e-487f-9c40-b07efc0fb171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.786116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9d2b9af7-997e-487f-9c40-b07efc0fb171 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:48.788241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9d2b9af7-997e-487f-9c40-b07efc0fb171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:48.797835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.800185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.815454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1509d-cb00-467e-b934-d1467ddaca8e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.851384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976ec36-3205-43c3-813c-21689ee5d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.851997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976ec36-3205-43c3-813c-21689ee5d062 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.852273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976ec36-3205-43c3-813c-21689ee5d062 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.852545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976ec36-3205-43c3-813c-21689ee5d062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.860715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 405/813] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:48.861353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b9af7-997e-487f-9c40-b07efc0fb171 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.938510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976ec36-3205-43c3-813c-21689ee5d062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.940103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976ec36-3205-43c3-813c-21689ee5d062 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.974259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b9af7-997e-487f-9c40-b07efc0fb171 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.009506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e59dc-8875-48e7-883d-6c34af05a230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.010413 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e59dc-8875-48e7-883d-6c34af05a230 None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.010841 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e59dc-8875-48e7-883d-6c34af05a230 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.011281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e59dc-8875-48e7-883d-6c34af05a230 None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/cf591850b9914f2f9089c8b6df730b4a/roles/1b16ca28088b4fb4a1e9cc15fea47675` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.029160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e59dc-8875-48e7-883d-6c34af05a230 None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=cf591850b9914f2f9089c8b6df730b4a, role_id=1b16ca28088b4fb4a1e9cc15fea47675)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.030977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.031142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e59dc-8875-48e7-883d-6c34af05a230 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.033724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.047383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 406/814] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:48 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/cf591850b9914f2f9089c8b6df730b4a/roles/1b16ca28088b4fb4a1e9cc15fea47675 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:49.047383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e59dc-8875-48e7-883d-6c34af05a230 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.057554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 409/815] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:46:49.058412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976ec36-3205-43c3-813c-21689ee5d062 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.177755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976ec36-3205-43c3-813c-21689ee5d062 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.190844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e59dc-8875-48e7-883d-6c34af05a230 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.223916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.224873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.225214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.225552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.230432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.230432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.233013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.233722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.234008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.234280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.234945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:49.235250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.236758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:49.239964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:49.241074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:49.241961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 410/816] 10.222.0.52 () {56 vars in 1129 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:49.242791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.250298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 407/817] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:49.251194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.295451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.313745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.348008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25725-cdf7-4591-988b-35638a074fc6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.362770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f05b98-961f-4387-b53a-774f37a77ccc None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.385215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.386188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.386674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.387208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.402335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.402335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.405566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.406216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.406501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.406777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] PATH_INFO: `/v3/services/network` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.407405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.408840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:49.418013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:49.419050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:49.419867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 408/818] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:49.420716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:49.422607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:49.423561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:49.424447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:49.424447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:49.424447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:49.427237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 411/819] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:49.428163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.531287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2d7d8-cfd4-4eb0-9273-981f5503189f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.536118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.539910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb8e05-0583-475a-a936-0ee2bc03d960 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.550051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.569860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.570689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.571140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.571410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.572037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:49.572331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.573692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.582921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 409/820] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:49.582921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.585358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.585908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.586184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.586453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.593701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.595156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:49.606417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:49.607507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 412/821] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:49.613391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.690428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dcff6-0e12-47a5-ae8f-c13787635634 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.716381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869454-e328-42ac-ae8d-e8bdf2fd4405 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.724601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.725296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.725638 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.725977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.726672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] RBAC: Adding query filter params (name=network) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:49.727008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.728406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.732861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 410/822] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:49.733629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.761180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.761956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.762283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.762612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.763340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:49.763703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.765200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.777577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 413/823] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:49.782873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.783201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.806775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.842654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16cef6-0090-4fde-8028-9b03f0827829 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.876309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.876916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.877196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.877462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.878083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:49.878380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.879794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.889093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 411/824] 10.222.0.52 () {56 vars in 1124 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:49.889093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.900763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e908ef-69cd-49cb-a4da-b315bca27d63 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.941593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.942991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.943233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.943504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.944215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:49.944518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.945956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.958908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 414/825] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:49.963944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.001269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5afd3-a554-4b2d-bc29-a72f177d8ed7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.012255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.037463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.038710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.039054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.039328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.039989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] RBAC: Adding query filter params (type=network) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:50.040292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.041714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.051377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 412/826] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:50.051377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.055133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.075753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07aaa8-a26c-44b5-b575-cb313a2a54ac None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.108818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.109699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.109986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.110417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.117502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.118952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:50.130742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:50.131905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:50.132809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 415/827] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:50.133327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.157565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba26d8-babf-4932-b6eb-004d03ab5f1c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.198102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953777-e68c-4f8e-b18e-48e058081e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.199004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953777-e68c-4f8e-b18e-48e058081e62 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.199288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953777-e68c-4f8e-b18e-48e058081e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.199560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953777-e68c-4f8e-b18e-48e058081e62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.200384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb953777-e68c-4f8e-b18e-48e058081e62 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.202305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb953777-e68c-4f8e-b18e-48e058081e62 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.225834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 413/828] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:46:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:50.226996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953777-e68c-4f8e-b18e-48e058081e62 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.251710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd0e87-be78-4ea0-ab2e-f3df454e66bc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.271636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.311884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4833fe9-9d04-4763-8e77-9cbef1121dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.313451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4833fe9-9d04-4763-8e77-9cbef1121dd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.314460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4833fe9-9d04-4763-8e77-9cbef1121dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.314885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4833fe9-9d04-4763-8e77-9cbef1121dd4 None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/91df91b1ea4949119245c86bf52e32f6/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.318411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4833fe9-9d04-4763-8e77-9cbef1121dd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=91df91b1ea4949119245c86bf52e32f6, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.321133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4833fe9-9d04-4763-8e77-9cbef1121dd4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.326639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.342354 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 416/829] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:50 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/91df91b1ea4949119245c86bf52e32f6/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:50.344047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4833fe9-9d04-4763-8e77-9cbef1121dd4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.366082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953777-e68c-4f8e-b18e-48e058081e62 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.462513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06028a26-ad0b-4e84-9135-4e64576a09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.463657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06028a26-ad0b-4e84-9135-4e64576a09da None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.464238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06028a26-ad0b-4e84-9135-4e64576a09da None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.464729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06028a26-ad0b-4e84-9135-4e64576a09da None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.465607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06028a26-ad0b-4e84-9135-4e64576a09da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.467332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06028a26-ad0b-4e84-9135-4e64576a09da None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.481188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4833fe9-9d04-4763-8e77-9cbef1121dd4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.524421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf556e5-88ab-45ce-9836-520aeab576ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.525290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf556e5-88ab-45ce-9836-520aeab576ba None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.525576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf556e5-88ab-45ce-9836-520aeab576ba None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.525963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf556e5-88ab-45ce-9836-520aeab576ba None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.526755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf556e5-88ab-45ce-9836-520aeab576ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:50.527071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf556e5-88ab-45ce-9836-520aeab576ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.528525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf556e5-88ab-45ce-9836-520aeab576ba None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.530740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 414/830] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:46:50 2022] POST /identity/v3/endpoints => generated 337 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:50.544853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 417/831] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:50.552682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06028a26-ad0b-4e84-9135-4e64576a09da None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.654634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.683519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06028a26-ad0b-4e84-9135-4e64576a09da None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.730660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd295a88-0332-4c3d-b587-e66c6fa3bb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.731410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd295a88-0332-4c3d-b587-e66c6fa3bb53 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.731695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd295a88-0332-4c3d-b587-e66c6fa3bb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.731965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd295a88-0332-4c3d-b587-e66c6fa3bb53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.732698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd295a88-0332-4c3d-b587-e66c6fa3bb53 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=152edb19fd7b434485b0e1ebc5078335) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:50.733003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd295a88-0332-4c3d-b587-e66c6fa3bb53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.734448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd295a88-0332-4c3d-b587-e66c6fa3bb53 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.742394 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 415/832] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:50 2022] GET /identity/v3/role_assignments?user.id=152edb19fd7b434485b0e1ebc5078335&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:51.285394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a439ff-d728-4011-97e6-0d1e0e15bf40 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:51.285780 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a439ff-d728-4011-97e6-0d1e0e15bf40 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:51.286478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a439ff-d728-4011-97e6-0d1e0e15bf40 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:51.288092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 418/833] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:51.293435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0829e-f224-493a-91f0-8c3628f1c83b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:51.293733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0829e-f224-493a-91f0-8c3628f1c83b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:51.294007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0829e-f224-493a-91f0-8c3628f1c83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:51.313320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-08e0829e-f224-493a-91f0-8c3628f1c83b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:51.314504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-08e0829e-f224-493a-91f0-8c3628f1c83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:51.387552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 416/834] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:51.671215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a439ff-d728-4011-97e6-0d1e0e15bf40 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:51.734062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:51.784952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a439ff-d728-4011-97e6-0d1e0e15bf40 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:51.820182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe7a3-e6a0-4c11-a7f1-30f4a39e8a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:51.820808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe7a3-e6a0-4c11-a7f1-30f4a39e8a25 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:51.821088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe7a3-e6a0-4c11-a7f1-30f4a39e8a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:51.821361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe7a3-e6a0-4c11-a7f1-30f4a39e8a25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:51.831608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe7a3-e6a0-4c11-a7f1-30f4a39e8a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-eebbe7a3-e6a0-4c11-a7f1-30f4a39e8a25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:51.836467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:51.837380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:51.838232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 419/835] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:51.843513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0829e-f224-493a-91f0-8c3628f1c83b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:51.920476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:51.945527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0829e-f224-493a-91f0-8c3628f1c83b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:51.980306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:51.980907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:51.981187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:51.981505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:51.982168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:51.982475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:51.983862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:51.992900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 417/836] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:51.998628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe7a3-e6a0-4c11-a7f1-30f4a39e8a25 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.101917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe7a3-e6a0-4c11-a7f1-30f4a39e8a25 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.119289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:52.137288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.137923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.138206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.138473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.148913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.150348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:52.165525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:52.166449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:52.167346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:52.167346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:52.167346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:52.167346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 420/837] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:52.172301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.279959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43bb28-8cfa-4e56-9fff-3c1e48ff28e5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.314677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.315327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.315605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.315872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.316505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:52.316801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.318790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.325975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:52.337096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.338136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 418/838] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:52.441892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa0d99-06c3-48ab-8504-0491187e4577 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.476630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.477251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.477530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.477798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.485446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.486897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:52.498514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:52.499528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:52.500466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 421/839] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:52.507780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.618395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01295cad-a7f1-4e5c-be53-1a776cf78446 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.655807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa4c34-1f8c-4d09-9957-18e65f86d7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.656786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4c34-1f8c-4d09-9957-18e65f86d7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.657284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4c34-1f8c-4d09-9957-18e65f86d7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.657761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4c34-1f8c-4d09-9957-18e65f86d7f5 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.658685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa4c34-1f8c-4d09-9957-18e65f86d7f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:52.659242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa4c34-1f8c-4d09-9957-18e65f86d7f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.661420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa4c34-1f8c-4d09-9957-18e65f86d7f5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.680349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 419/840] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:52.686310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.713954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03bdaf-56bc-431d-b137-13df6dcbf244 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.714623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03bdaf-56bc-431d-b137-13df6dcbf244 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.715136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03bdaf-56bc-431d-b137-13df6dcbf244 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.717278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 420/841] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:52.726609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:52.728955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f460d-8a6d-4506-84c7-0c6908950819 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.731327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f460d-8a6d-4506-84c7-0c6908950819 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.731327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f460d-8a6d-4506-84c7-0c6908950819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.748823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b23f460d-8a6d-4506-84c7-0c6908950819 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:52.750145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b23f460d-8a6d-4506-84c7-0c6908950819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:52.810479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63c66-adad-4d1e-9085-3ef06ed13db7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.814432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:52.839588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 421/842] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:52.852079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfddc7-921b-4d54-aa48-5d0e2388808b None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.852972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfddc7-921b-4d54-aa48-5d0e2388808b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.853388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfddc7-921b-4d54-aa48-5d0e2388808b None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.855249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 422/843] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:52.860298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74ad54-34e8-4091-91f4-9f0e32effb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.860682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26a615-9052-49db-b1e2-ad419fcb61ee None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.860906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74ad54-34e8-4091-91f4-9f0e32effb1e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.861185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74ad54-34e8-4091-91f4-9f0e32effb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.861452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74ad54-34e8-4091-91f4-9f0e32effb1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.861715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26a615-9052-49db-b1e2-ad419fcb61ee None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.862168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26a615-9052-49db-b1e2-ad419fcb61ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.883749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-1a26a615-9052-49db-b1e2-ad419fcb61ee None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:52.885102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1a26a615-9052-49db-b1e2-ad419fcb61ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:52.949153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74ad54-34e8-4091-91f4-9f0e32effb1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.950977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74ad54-34e8-4091-91f4-9f0e32effb1e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.974671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 423/844] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:46:52.991302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.061217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 422/845] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:46:53.066822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26a615-9052-49db-b1e2-ad419fcb61ee None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.103838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6c8b9-52a1-479d-adbb-f97707df92e2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.104161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6c8b9-52a1-479d-adbb-f97707df92e2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.104384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6c8b9-52a1-479d-adbb-f97707df92e2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.105831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 423/846] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:53.111428 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f44c6d-620b-4dfd-a75e-0ecd5e983824 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.111940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f44c6d-620b-4dfd-a75e-0ecd5e983824 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.112436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f44c6d-620b-4dfd-a75e-0ecd5e983824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.130719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-63f44c6d-620b-4dfd-a75e-0ecd5e983824 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:53.132134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-63f44c6d-620b-4dfd-a75e-0ecd5e983824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:53.182014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26a615-9052-49db-b1e2-ad419fcb61ee None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.199695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.206157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 424/847] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:53.206817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f44c6d-620b-4dfd-a75e-0ecd5e983824 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.231307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885179db-5e64-4e00-a692-325c9478937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.232630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885179db-5e64-4e00-a692-325c9478937a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.233499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885179db-5e64-4e00-a692-325c9478937a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.234233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885179db-5e64-4e00-a692-325c9478937a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.246429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885179db-5e64-4e00-a692-325c9478937a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.247879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885179db-5e64-4e00-a692-325c9478937a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-885179db-5e64-4e00-a692-325c9478937a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:53.262110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.263074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:53.264416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:53.264416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:53.264416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:53.264416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 424/848] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:53.264416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885179db-5e64-4e00-a692-325c9478937a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.273480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.319610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f44c6d-620b-4dfd-a75e-0ecd5e983824 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.355062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1e86f-9714-4f68-b77e-8de64b5b1431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.356105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1e86f-9714-4f68-b77e-8de64b5b1431 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.356396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1e86f-9714-4f68-b77e-8de64b5b1431 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.356825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1e86f-9714-4f68-b77e-8de64b5b1431 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.367571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1e86f-9714-4f68-b77e-8de64b5b1431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:53.368172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885179db-5e64-4e00-a692-325c9478937a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9cf1e86f-9714-4f68-b77e-8de64b5b1431 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:53.372314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.373226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:53.374054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:53.378340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 425/849] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:53.379184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1e86f-9714-4f68-b77e-8de64b5b1431 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.409763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.410405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.410690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.411542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.412127 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.413497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.425677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.431087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:53.431966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 425/850] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:53.432790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.460131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.502964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1e86f-9714-4f68-b77e-8de64b5b1431 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.519648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.528566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abaca4-eb1e-4456-923c-22213ee98e53 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.538616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.539252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.539539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.539809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.540450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:53.540751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.542097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.554307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 426/851] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:53.555140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.566333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.566972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.567782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.568055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.568712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:53.569012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.570362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.582280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 426/852] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:53.584233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.657290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f02b1-7ce2-4f00-8fa5-db4a31028211 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.674313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.681681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f753a-a984-4dfe-8762-022c94df69a4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.689922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.690464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.690732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.691027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.691645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:53.691940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.693778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.698027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 427/853] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:53.698783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.716757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.717375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.717653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.717918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.718492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.719851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.728414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.730173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:53.732326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 427/854] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:53.733943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.742861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.797512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458f9e1-7948-48cf-96fb-4d56be1e8b48 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.829784 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd30075-d5b9-4568-92dc-c4286adc9c51 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.833459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.834082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.834364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.834635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.841124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.842545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:53.853375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.854282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 428/855] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:53.855213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.863414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.864589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.864865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.865130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.874813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.876285 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:53.891062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.892158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:53.893022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:53.893022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:53.893022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:53.893022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 428/856] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:53.893259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.907483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.953841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529f47d-758d-4ff6-82e4-8f73207b2512 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.955324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.988416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.989550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.989842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.990111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.990736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:53.991061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.992416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.993161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0c8df-1363-49af-9acb-fcd0eb933ed8 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.997398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 429/857] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:53.998244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.032156 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.032778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.033054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.033322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.033961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.034263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.035699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.041367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 429/858] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.046753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.117760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae569062-62b9-4cee-96fb-dca2caee50a0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.128237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.155861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.156482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.156830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.157075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.157735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.158037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.158946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70d97-48db-42c9-a722-605f2fbe3b8b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.159512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.174776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 430/859] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.175659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.202300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.203053 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.203337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.203611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.204306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.204546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.205900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.206743 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.222524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 430/860] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.222524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.292708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639f16-3b0d-4caf-8071-bdfaa9b74b14 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.325073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc5f9d-6614-4714-9a90-220a20b2dc7f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.332107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.333283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.333847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.333980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.334498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.334792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.336173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.340938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 431/861] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.351947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.362813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.363547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.363865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.364217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.365003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=cf591850b9914f2f9089c8b6df730b4a) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.365307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.366785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.369120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.383370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 431/862] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/role_assignments?user.id=cf591850b9914f2f9089c8b6df730b4a&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.482467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.496916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3e936-2860-4ac6-91ec-ebc31abe9947 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.539197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.540253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.540750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.541240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.549820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.551936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:54.565852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:54.566976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:54.568551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 432/863] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:54.573398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.683873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b7a0c-f544-4f97-9859-434bedaf1f7f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.720271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.720888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.721165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.721436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.722174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.722480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.723989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.744458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 432/864] 10.222.0.52 () {56 vars in 1135 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.749805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.856313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7998405-b6fe-4a8d-993f-3f536562c2e5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.892231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a066c92-8343-4581-b7f3-d8e46ef4dbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.892865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a066c92-8343-4581-b7f3-d8e46ef4dbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.893153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a066c92-8343-4581-b7f3-d8e46ef4dbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.893430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a066c92-8343-4581-b7f3-d8e46ef4dbe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.915547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.966484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a066c92-8343-4581-b7f3-d8e46ef4dbe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.967976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a066c92-8343-4581-b7f3-d8e46ef4dbe8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.070912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 433/865] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:46:55.076163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.136297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.182872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130a5dc-0dcc-4c1b-9917-0c8fc872a62b None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.217540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.218184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.218463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.218738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.229843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.231309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:55.246765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:55.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:55.248624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:55.248624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:55.248624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:55.248624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 433/866] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:55.255997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a066c92-8343-4581-b7f3-d8e46ef4dbe8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.309303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.360062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a066c92-8343-4581-b7f3-d8e46ef4dbe8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.393751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.394346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.394620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.394903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.396232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:55.396532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.397875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.409476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 434/867] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:55.415634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.524947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b4ad-c306-47ff-b62a-7ebe35a841c4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.543221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.561703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.562327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.562854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.563145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.570509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.572040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:55.583449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:55.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:55.585561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 434/868] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:55.590549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.696603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b956e-7b15-40d1-8a78-abc7a9af6af3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.706771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.731059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce000af-5bcc-4531-a063-0f3c5964d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.731666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce000af-5bcc-4531-a063-0f3c5964d2af None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.731937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce000af-5bcc-4531-a063-0f3c5964d2af None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.732262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce000af-5bcc-4531-a063-0f3c5964d2af None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.732953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce000af-5bcc-4531-a063-0f3c5964d2af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:55.733253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce000af-5bcc-4531-a063-0f3c5964d2af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.734657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce000af-5bcc-4531-a063-0f3c5964d2af None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.749061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 435/869] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:55.755384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.857881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43dc2c-03bc-4a76-9973-3d657808a16d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.891822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992cbc9-1297-4481-9b16-cf16df19b9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.892415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992cbc9-1297-4481-9b16-cf16df19b9ba None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.892689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992cbc9-1297-4481-9b16-cf16df19b9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.892953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992cbc9-1297-4481-9b16-cf16df19b9ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.893666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992cbc9-1297-4481-9b16-cf16df19b9ba None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=91df91b1ea4949119245c86bf52e32f6) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:55.894025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992cbc9-1297-4481-9b16-cf16df19b9ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.900396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992cbc9-1297-4481-9b16-cf16df19b9ba None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.906376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 435/870] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/role_assignments?user.id=91df91b1ea4949119245c86bf52e32f6&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:56.201047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5c50-a3e0-4962-84d8-0f012448b8eb None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.201600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5c50-a3e0-4962-84d8-0f012448b8eb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.201600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5c50-a3e0-4962-84d8-0f012448b8eb None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.203180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 436/871] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:56.209074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf69cb7-3e5a-46ba-8743-b6b6342f0c97 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.209536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf69cb7-3e5a-46ba-8743-b6b6342f0c97 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.209631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf69cb7-3e5a-46ba-8743-b6b6342f0c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.228608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-1bf69cb7-3e5a-46ba-8743-b6b6342f0c97 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:56.229781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1bf69cb7-3e5a-46ba-8743-b6b6342f0c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:56.256238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:56.300182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 436/872] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:56.579591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5c50-a3e0-4962-84d8-0f012448b8eb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:56.585978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec43036-f932-41a6-9ccd-9ba992eedad2 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.586261 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec43036-f932-41a6-9ccd-9ba992eedad2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.586532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec43036-f932-41a6-9ccd-9ba992eedad2 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.588131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 437/873] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:56.593294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0f4ec-d95c-479f-98bc-776fac6d4c4e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.593580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0f4ec-d95c-479f-98bc-776fac6d4c4e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.593855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0f4ec-d95c-479f-98bc-776fac6d4c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.620065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f2c0f4ec-d95c-479f-98bc-776fac6d4c4e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:56.621633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f2c0f4ec-d95c-479f-98bc-776fac6d4c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:56.695816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 438/874] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:46:56.706698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5c50-a3e0-4962-84d8-0f012448b8eb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.747549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0d566-f7a9-4cf9-a6d2-d5caf31cf569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.748161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0d566-f7a9-4cf9-a6d2-d5caf31cf569 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.748438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0d566-f7a9-4cf9-a6d2-d5caf31cf569 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.748756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0d566-f7a9-4cf9-a6d2-d5caf31cf569 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.749422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0d566-f7a9-4cf9-a6d2-d5caf31cf569 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.750759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0d566-f7a9-4cf9-a6d2-d5caf31cf569 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.769987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 437/875] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:46:56 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:56.918763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977433-5d0e-4553-b83c-47c4946474c3 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.918990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977433-5d0e-4553-b83c-47c4946474c3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.919344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977433-5d0e-4553-b83c-47c4946474c3 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.921011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 439/876] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:56.927269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea384b1-5d5f-434d-9aa4-3bb8d6f337ac None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.927574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea384b1-5d5f-434d-9aa4-3bb8d6f337ac None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.927830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea384b1-5d5f-434d-9aa4-3bb8d6f337ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.947893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4ea384b1-5d5f-434d-9aa4-3bb8d6f337ac None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:56.949101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4ea384b1-5d5f-434d-9aa4-3bb8d6f337ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:56.975920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:56.986125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db977433-5d0e-4553-b83c-47c4946474c3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.047010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 438/877] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:57.097909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:57.108767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db977433-5d0e-4553-b83c-47c4946474c3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.144713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155adfa-c8dc-4618-82bb-b19b1af4b0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.146246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155adfa-c8dc-4618-82bb-b19b1af4b0bc None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.146754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155adfa-c8dc-4618-82bb-b19b1af4b0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.147257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155adfa-c8dc-4618-82bb-b19b1af4b0bc None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.148120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155adfa-c8dc-4618-82bb-b19b1af4b0bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.149700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155adfa-c8dc-4618-82bb-b19b1af4b0bc None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.168746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 440/878] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:46:56 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:46:57.322208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea384b1-5d5f-434d-9aa4-3bb8d6f337ac None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.424566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea384b1-5d5f-434d-9aa4-3bb8d6f337ac None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.464744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedac91-777e-4015-ba59-1e07cc297d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.465372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedac91-777e-4015-ba59-1e07cc297d83 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.465646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedac91-777e-4015-ba59-1e07cc297d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.465914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedac91-777e-4015-ba59-1e07cc297d83 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.476678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedac91-777e-4015-ba59-1e07cc297d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5fedac91-777e-4015-ba59-1e07cc297d83 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:57.481347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:57.482337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:57.483262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 439/879] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:57.488441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155adfa-c8dc-4618-82bb-b19b1af4b0bc None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.595722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155adfa-c8dc-4618-82bb-b19b1af4b0bc None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.637560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.638183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.638465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.638736 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.639436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:57.639740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.641112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.654464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 441/880] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:57.656000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedac91-777e-4015-ba59-1e07cc297d83 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.671781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:57.766798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedac91-777e-4015-ba59-1e07cc297d83 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.805015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.806058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.806549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.807077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.818321 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.820052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:57.835555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:57.836502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:57.837387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:57.837387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:57.837387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:57.839436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 440/881] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:57.848334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.865264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:57.957268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674f68f-90c4-4dbf-8d30-5e60816c0b75 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.993648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3e85b-66d8-4f0a-898b-f56e3082ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.994505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e85b-66d8-4f0a-898b-f56e3082ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.994991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e85b-66d8-4f0a-898b-f56e3082ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.995427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e85b-66d8-4f0a-898b-f56e3082ffa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.996236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e85b-66d8-4f0a-898b-f56e3082ffa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:57.996695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e85b-66d8-4f0a-898b-f56e3082ffa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.998217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e85b-66d8-4f0a-898b-f56e3082ffa9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.015042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 442/882] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:58.017310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.088192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.137813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b537e-cfd5-4fc3-8e62-80bce977a5f9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.171986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.172611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.172887 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.173150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.181455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.182911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.190599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308423-b60b-42dd-a2e5-af41387d5e01 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.191169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308423-b60b-42dd-a2e5-af41387d5e01 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.191666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308423-b60b-42dd-a2e5-af41387d5e01 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.193547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 443/883] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:58.194278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:58.195250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:58.198258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 441/884] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:58.201348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75c4ce-96e0-4226-a30a-9f2191775a7b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.203440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75c4ce-96e0-4226-a30a-9f2191775a7b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.203440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75c4ce-96e0-4226-a30a-9f2191775a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.204531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.221198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-0d75c4ce-96e0-4226-a30a-9f2191775a7b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:58.222495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0d75c4ce-96e0-4226-a30a-9f2191775a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:58.306673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 444/885] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:58.324285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ec98c-e7c7-43f1-b18c-48573b6b998b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.349380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.360863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.361528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.362016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.362290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.363114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:58.363422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.364926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.382806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 442/886] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:58.388301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75c4ce-96e0-4226-a30a-9f2191775a7b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.392041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.504568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75c4ce-96e0-4226-a30a-9f2191775a7b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.542005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74938b-268f-484d-bdf4-a936c949427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.542609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74938b-268f-484d-bdf4-a936c949427f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.542883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74938b-268f-484d-bdf4-a936c949427f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.543171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74938b-268f-484d-bdf4-a936c949427f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.595454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.611889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74938b-268f-484d-bdf4-a936c949427f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.613328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74938b-268f-484d-bdf4-a936c949427f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.616838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.714622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ca78a-864b-47a9-826d-5495c185e716 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.747060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 445/887] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:46:58.750853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74938b-268f-484d-bdf4-a936c949427f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.760889 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.761519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.761794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.762063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.763119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.764476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:58.775553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:58.776581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:46:58.777415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:58.782027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 443/888] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:58.782876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.841823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.882378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74938b-268f-484d-bdf4-a936c949427f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.885769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.902204 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3226d21-2e28-4802-9ce1-f94a3fa4ab8d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.927676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.928888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.929350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.929811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.943501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.945111 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.949004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.949844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.950305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.950752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.951637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:58.952187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.953737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.959830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 444/889] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:58.964826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:58.965765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:58.966650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:58.966650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:58.966650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:58.966816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.967305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 446/890] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:58.975858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.094928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb2426-f229-4186-8acb-41c3ccc5a6fe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.107908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.111372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81e3bc-bcbc-4383-9b0d-1894cab57736 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.142193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab496d92-6472-48f1-8dbf-38f89eaeb5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.143471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab496d92-6472-48f1-8dbf-38f89eaeb5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.144139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab496d92-6472-48f1-8dbf-38f89eaeb5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.144586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab496d92-6472-48f1-8dbf-38f89eaeb5b7 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.145893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab496d92-6472-48f1-8dbf-38f89eaeb5b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:59.146337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab496d92-6472-48f1-8dbf-38f89eaeb5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.147868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab496d92-6472-48f1-8dbf-38f89eaeb5b7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.148302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0757aeb-fa8b-40be-b86c-fbb500e9e9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.148917 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757aeb-fa8b-40be-b86c-fbb500e9e9bb None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.149202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757aeb-fa8b-40be-b86c-fbb500e9e9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.149477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757aeb-fa8b-40be-b86c-fbb500e9e9bb None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.150124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0757aeb-fa8b-40be-b86c-fbb500e9e9bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:59.150424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0757aeb-fa8b-40be-b86c-fbb500e9e9bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.151810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0757aeb-fa8b-40be-b86c-fbb500e9e9bb None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.155167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 445/891] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:59.164332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde696bf-03f5-4344-8c74-1aade4a23438 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.165603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde696bf-03f5-4344-8c74-1aade4a23438 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.166173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde696bf-03f5-4344-8c74-1aade4a23438 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.172385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 447/893] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:59.172634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 446/893] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:59.174750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde696bf-03f5-4344-8c74-1aade4a23438 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.175089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4830ca-f169-48f3-8fc6-c39748b176ee None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.175522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4830ca-f169-48f3-8fc6-c39748b176ee None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.175962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4830ca-f169-48f3-8fc6-c39748b176ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.199068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-8e4830ca-f169-48f3-8fc6-c39748b176ee None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:59.200250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8e4830ca-f169-48f3-8fc6-c39748b176ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:59.213710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.283001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 448/894] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:46:59.318507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde696bf-03f5-4344-8c74-1aade4a23438 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.362521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d6a7b-e5c3-4dbe-a730-63c6867bde44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.363244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d6a7b-e5c3-4dbe-a730-63c6867bde44 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.363518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d6a7b-e5c3-4dbe-a730-63c6867bde44 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.363968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d6a7b-e5c3-4dbe-a730-63c6867bde44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.371172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d6a7b-e5c3-4dbe-a730-63c6867bde44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.372612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d6a7b-e5c3-4dbe-a730-63c6867bde44 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b18d6a7b-e5c3-4dbe-a730-63c6867bde44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:59.384976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:59.385867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:59.386753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:59.386753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:59.386753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:59.386753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:59.386753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:59.386753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:59.386753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:59.386753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:46:59.390884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 447/895] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:59.392300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4830ca-f169-48f3-8fc6-c39748b176ee None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.454025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa05eb-3f96-48ec-9cad-0fe2d26c3272 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.454303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa05eb-3f96-48ec-9cad-0fe2d26c3272 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.454577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa05eb-3f96-48ec-9cad-0fe2d26c3272 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.456203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 448/896] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:59.461806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1867a8-6ffc-4976-926f-bc85bcb821f0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.461976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1867a8-6ffc-4976-926f-bc85bcb821f0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.462255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1867a8-6ffc-4976-926f-bc85bcb821f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.480213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-af1867a8-6ffc-4976-926f-bc85bcb821f0 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:59.481419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-af1867a8-6ffc-4976-926f-bc85bcb821f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:59.503752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4830ca-f169-48f3-8fc6-c39748b176ee None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.509839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.542614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.543259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.543487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.543749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.544558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:59.544869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.546298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.557150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 449/897] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:46:59.565313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 449/898] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:59.576034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1867a8-6ffc-4976-926f-bc85bcb821f0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.587704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.597644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.690002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1867a8-6ffc-4976-926f-bc85bcb821f0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.702808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a14f8-8897-444b-b3c9-242cdac23ec7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.731147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.731909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.732289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.732914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.734140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] RBAC: Adding query filter params (role.id=2b49ddcd3d3346a4879bbf33c27d9419, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=cf591850b9914f2f9089c8b6df730b4a) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:59.734550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.736574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.739520 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.740328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.740829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.741310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.742078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.749192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 450/899] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/role_assignments?user.id=cf591850b9914f2f9089c8b6df730b4a&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2b49ddcd3d3346a4879bbf33c27d9419 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:59.749387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:59.757964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:59.759050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:46:59.760286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 450/900] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:59.765546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.798738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.832147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.895598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e069d-128e-4a0e-946a-7f07c19fb6c7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.920942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.944815 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.945797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.946301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.946791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.947761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:59.948312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.949896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.955582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 451/901] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:59.959939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6904e-6392-4c79-be86-20a84f5457cf None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.960997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.002588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.003620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.004105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.005143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.006452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.008440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:00.021251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:00.022228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:00.023120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:00.029375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 451/902] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:00.030202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.069588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.079619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24653b-43ff-4109-8334-36a972a08543 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.121078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.121681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.121964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.122247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.122926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:00.123244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.124605 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.130515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 452/903] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:00.135843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.179108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c3618-93d3-4938-afe7-6e50c448c996 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.224890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.226286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.227187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.227754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.228816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:00.229300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.231672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.233337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.240344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 452/904] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:00.246517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.261361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b2da7-92f4-47ea-ba6f-7aa0c2b89442 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.302467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.303469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.303967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.304448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.305408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] RBAC: Adding query filter params (interface=public, service_id=210867b7428c4f3293b67df710e853d3, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:00.305931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.307784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.323120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 453/905] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/endpoints?service_id=210867b7428c4f3293b67df710e853d3&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:00.325128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.352636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.356219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d81cee-37d6-4fb0-9779-6d291a2526f9 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.401684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.402667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.403172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.403661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.405224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:00.406038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.407697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.420641 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 453/906] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:00.422566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.452295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718287-3e2a-4711-aa3b-feb083feb2be None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.493572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff89028-ccff-441a-b2e0-ea7015bc59d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.494185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff89028-ccff-441a-b2e0-ea7015bc59d3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.494569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff89028-ccff-441a-b2e0-ea7015bc59d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.494843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff89028-ccff-441a-b2e0-ea7015bc59d3 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.495466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff89028-ccff-441a-b2e0-ea7015bc59d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.496825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff89028-ccff-441a-b2e0-ea7015bc59d3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.510931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 454/907] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:47:00.513586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.557974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b60e20-96cd-45f1-837e-751edb40bd71 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.596196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc935-3d0f-4968-b8fa-6dfc78e2286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.596734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc935-3d0f-4968-b8fa-6dfc78e2286d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.597015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc935-3d0f-4968-b8fa-6dfc78e2286d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.597283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc935-3d0f-4968-b8fa-6dfc78e2286d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.597993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc935-3d0f-4968-b8fa-6dfc78e2286d None admin] RBAC: Adding query filter params (interface=public, service_id=73e059dbe4604230bc527906c6c7fb41, region_id=RegionOne) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:00.598868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc935-3d0f-4968-b8fa-6dfc78e2286d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.600520 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc935-3d0f-4968-b8fa-6dfc78e2286d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.606697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 454/908] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/endpoints?service_id=73e059dbe4604230bc527906c6c7fb41&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:00.612606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff89028-ccff-441a-b2e0-ea7015bc59d3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.713298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.730201 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff89028-ccff-441a-b2e0-ea7015bc59d3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.764499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49624a-9ddf-474a-b42e-2249fca0d744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.765104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49624a-9ddf-474a-b42e-2249fca0d744 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.765386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49624a-9ddf-474a-b42e-2249fca0d744 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.765720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49624a-9ddf-474a-b42e-2249fca0d744 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.766597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49624a-9ddf-474a-b42e-2249fca0d744 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.767716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49624a-9ddf-474a-b42e-2249fca0d744 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.774462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 455/909] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:47:01.417506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242513-4fa5-4af3-b362-e07a34d711eb None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.417797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242513-4fa5-4af3-b362-e07a34d711eb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.418065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242513-4fa5-4af3-b362-e07a34d711eb None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.419731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 455/910] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:01.425185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff93431-80f7-4831-b58c-c7c23cff6ac4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.425559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff93431-80f7-4831-b58c-c7c23cff6ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.425846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff93431-80f7-4831-b58c-c7c23cff6ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.445956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7ff93431-80f7-4831-b58c-c7c23cff6ac4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:01.447197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7ff93431-80f7-4831-b58c-c7c23cff6ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:01.521703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 456/911] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:01.790536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3242513-4fa5-4af3-b362-e07a34d711eb None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:01.902418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3242513-4fa5-4af3-b362-e07a34d711eb None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:01.929355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:01.936484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2371a4-78cd-4386-a656-90f41af3cd85 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.936772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2371a4-78cd-4386-a656-90f41af3cd85 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.937045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2371a4-78cd-4386-a656-90f41af3cd85 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.939474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16486cb5-6fe6-462d-99ac-0a59efb522dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:01.939850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 457/912] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:01.940076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16486cb5-6fe6-462d-99ac-0a59efb522dd None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.940361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16486cb5-6fe6-462d-99ac-0a59efb522dd None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.940697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16486cb5-6fe6-462d-99ac-0a59efb522dd None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.941351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16486cb5-6fe6-462d-99ac-0a59efb522dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:01.943275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16486cb5-6fe6-462d-99ac-0a59efb522dd None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:01.944216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b4fc5-e0a7-447f-bc1b-9c5c78de7f57 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.944503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b4fc5-e0a7-447f-bc1b-9c5c78de7f57 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.944779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b4fc5-e0a7-447f-bc1b-9c5c78de7f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.961098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 456/913] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:01 2022] POST /identity/v3/services => generated 241 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:01.968201 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-944b4fc5-e0a7-447f-bc1b-9c5c78de7f57 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:01.969526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-944b4fc5-e0a7-447f-bc1b-9c5c78de7f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:02.057886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 458/914] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:02.327646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16486cb5-6fe6-462d-99ac-0a59efb522dd None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:02.433770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16486cb5-6fe6-462d-99ac-0a59efb522dd None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.474038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a1087-f215-4d8b-9cd4-de306811dd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:02.474678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1087-f215-4d8b-9cd4-de306811dd0f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.474983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1087-f215-4d8b-9cd4-de306811dd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.475251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a1087-f215-4d8b-9cd4-de306811dd0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.485947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a1087-f215-4d8b-9cd4-de306811dd0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-c19a1087-f215-4d8b-9cd4-de306811dd0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:02.491792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:02.492710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:02.493533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 457/915] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Oct 10 11:47:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:02.499737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b4fc5-e0a7-447f-bc1b-9c5c78de7f57 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:02.502843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:02.605681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b4fc5-e0a7-447f-bc1b-9c5c78de7f57 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.641227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:02.641835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.642106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.642370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.643058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:02.643352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:02.644692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:02.653294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 459/916] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Oct 10 11:47:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:02.660266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a1087-f215-4d8b-9cd4-de306811dd0f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:02.709534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:02.770975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a1087-f215-4d8b-9cd4-de306811dd0f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.805977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36672b-526f-4095-bcb1-0585497c6c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:02.806729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36672b-526f-4095-bcb1-0585497c6c1e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.806912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36672b-526f-4095-bcb1-0585497c6c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.807147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36672b-526f-4095-bcb1-0585497c6c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.818564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36672b-526f-4095-bcb1-0585497c6c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:02.820044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36672b-526f-4095-bcb1-0585497c6c1e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-da36672b-526f-4095-bcb1-0585497c6c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:02.834506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:02.836720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:02.838749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:02.838749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:02.838749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:02.838749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 458/917] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:02.844103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:02.875938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac6ed-13f4-4d4a-a37a-5f3d7d0d19ce None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.876129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac6ed-13f4-4d4a-a37a-5f3d7d0d19ce None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.876517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac6ed-13f4-4d4a-a37a-5f3d7d0d19ce None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.877859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 459/918] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:02.883798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f434b5-c659-4f92-a0bf-564c336bd641 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.884085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f434b5-c659-4f92-a0bf-564c336bd641 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.884364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f434b5-c659-4f92-a0bf-564c336bd641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.905228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e1f434b5-c659-4f92-a0bf-564c336bd641 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:02.906583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e1f434b5-c659-4f92-a0bf-564c336bd641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:02.916989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:02.965083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822891a-83c4-4944-b8c7-21bee4e44d69 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.991452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 460/919] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:03.008633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.009241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.009516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.009786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.010477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:03.010777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.012202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.020633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf5c0b-0826-44a7-970e-90eb104449b5 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.022587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf5c0b-0826-44a7-970e-90eb104449b5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.022587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf5c0b-0826-44a7-970e-90eb104449b5 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.023753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 461/920] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:03.025900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 460/921] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:03.031327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d51a0-5b8e-4cb9-b3f2-9f48e3b7ac2b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.031612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d51a0-5b8e-4cb9-b3f2-9f48e3b7ac2b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.031885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d51a0-5b8e-4cb9-b3f2-9f48e3b7ac2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.035297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.060211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-484d51a0-5b8e-4cb9-b3f2-9f48e3b7ac2b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:03.061974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-484d51a0-5b8e-4cb9-b3f2-9f48e3b7ac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:03.089950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.145759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 462/922] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:03.154719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa7bbf-bd4c-4b85-b388-ed4d9e6da4bf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.183042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.192430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.193065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.193343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.193668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.201111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.202645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:03.214623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:03.215578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:03.216598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:03.221156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 461/923] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:03.222550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d51a0-5b8e-4cb9-b3f2-9f48e3b7ac2b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.286153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.328970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d51a0-5b8e-4cb9-b3f2-9f48e3b7ac2b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.363194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.364359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.364643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.364914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.365643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:03.365947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.367405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.385165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 463/924] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:03.390533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.411046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55469ff1-3da6-4946-a623-806fc1a794cd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.421945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.450312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.450981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.451270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.451546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.452134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.453618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:03.462951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:03.463935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:03.464801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 462/925] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:03.465589 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.514155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.518058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eb4ee-e794-46e2-a205-5f2d33e22d50 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.553649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.554488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.554769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.555069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.564903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e0e5e-3a03-4a7a-8224-d3e22731348f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.565317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.566818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:03.581060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:03.582013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:03.583470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:03.583470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:03.583470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:03.583470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 464/926] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:03.583470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.605797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.606423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.606708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.607009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.607584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.608923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:03.618126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:03.619094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:03.620011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 463/927] 10.222.0.52 () {56 vars in 1132 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:03.620782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.663865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.701740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac821a-0451-4019-8d2d-a1f4e0098e2e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.732304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bf89f-76c3-4416-9f2d-81ea8ef29c9b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.737300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.737908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.738188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.738460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.739139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:03.739442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.740802 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.745981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 465/928] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:03.746750 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.752287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.775793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.776684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.776966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.777235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.777881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:03.778182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.779559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.798644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 464/929] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:03.798644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.854509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393ea8-dab8-473e-949b-d08c3598ae4a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.886271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.892441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.893174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.893460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.893738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.894369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:03.894680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.896122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.905878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 466/930] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:03.906741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.911364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4dc24-ce6e-419a-bfa0-7072245556e9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.951257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.952140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.952422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.952692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.953685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:03.953948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.955460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.961119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 465/931] 10.222.0.52 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:03.961828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.002993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.019118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092c196-a90a-4673-94e0-a0dcda77cc7c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.055857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.056636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.056945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.057192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.064127 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.065549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.069550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf614fb-ee71-41cb-9ea0-d62122dc7252 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:04.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:04.078086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:04.079013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 467/932] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:04.079499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.109038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.109653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.109932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.110211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.110871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:04.111178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.112578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.119473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 466/933] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:04.120778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.148523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.195992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2641900-ea3e-410e-8dd9-1d4eda39c474 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.227028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af058d3b-d290-462d-a3d1-d912e8adace3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.228473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.229453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f96af8-965d-42db-b6e4-3cf265c430c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.230096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f96af8-965d-42db-b6e4-3cf265c430c6 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.230384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f96af8-965d-42db-b6e4-3cf265c430c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.230616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f96af8-965d-42db-b6e4-3cf265c430c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.231315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f96af8-965d-42db-b6e4-3cf265c430c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.232802 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f96af8-965d-42db-b6e4-3cf265c430c6 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.262920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 468/934] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:04.262920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f96af8-965d-42db-b6e4-3cf265c430c6 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.282821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6164f-7652-48cf-97ee-6576570d0663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.283478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6164f-7652-48cf-97ee-6576570d0663 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.283755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6164f-7652-48cf-97ee-6576570d0663 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.284065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6164f-7652-48cf-97ee-6576570d0663 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.284727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6164f-7652-48cf-97ee-6576570d0663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:04.285099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6164f-7652-48cf-97ee-6576570d0663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.287201 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6164f-7652-48cf-97ee-6576570d0663 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.300527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 467/935] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:04.302813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875c6c4-7001-4d9e-a04f-3972239bfd10 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.303346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875c6c4-7001-4d9e-a04f-3972239bfd10 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.303832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875c6c4-7001-4d9e-a04f-3972239bfd10 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.309408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 468/936] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:04.309408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875c6c4-7001-4d9e-a04f-3972239bfd10 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.370719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f96af8-965d-42db-b6e4-3cf265c430c6 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.385742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.415942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88a446-4737-4cdd-87e3-0f1b86826205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.416576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88a446-4737-4cdd-87e3-0f1b86826205 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.416906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88a446-4737-4cdd-87e3-0f1b86826205 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.417328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88a446-4737-4cdd-87e3-0f1b86826205 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.418132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88a446-4737-4cdd-87e3-0f1b86826205 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.419680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88a446-4737-4cdd-87e3-0f1b86826205 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.429496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875c6c4-7001-4d9e-a04f-3972239bfd10 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.446221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 469/937] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:04.451952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcade92b-0e6b-4482-a83c-9ae3f9bf6ce5 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.452252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcade92b-0e6b-4482-a83c-9ae3f9bf6ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.452629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcade92b-0e6b-4482-a83c-9ae3f9bf6ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.473094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-fcade92b-0e6b-4482-a83c-9ae3f9bf6ce5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:04.474360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fcade92b-0e6b-4482-a83c-9ae3f9bf6ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:04.482304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6e565-3ede-4aed-b7ad-eb85c1024755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.482967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e565-3ede-4aed-b7ad-eb85c1024755 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.483197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e565-3ede-4aed-b7ad-eb85c1024755 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.483471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e565-3ede-4aed-b7ad-eb85c1024755 None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/cf591850b9914f2f9089c8b6df730b4a/roles/2b49ddcd3d3346a4879bbf33c27d9419` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.485792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6e565-3ede-4aed-b7ad-eb85c1024755 None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=cf591850b9914f2f9089c8b6df730b4a, role_id=2b49ddcd3d3346a4879bbf33c27d9419)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.487286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6e565-3ede-4aed-b7ad-eb85c1024755 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.511186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 469/938] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:04 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/cf591850b9914f2f9089c8b6df730b4a/roles/2b49ddcd3d3346a4879bbf33c27d9419 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:47:04.564469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 470/939] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:04.844401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6e565-3ede-4aed-b7ad-eb85c1024755 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.862476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.953244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6e565-3ede-4aed-b7ad-eb85c1024755 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.988067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.989258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.989545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.989817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.990579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.992030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:05.001280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:05.002261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:05.003121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 470/940] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:05.008762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcade92b-0e6b-4482-a83c-9ae3f9bf6ce5 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.113861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.117946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcade92b-0e6b-4482-a83c-9ae3f9bf6ce5 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.154589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.155233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.155515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.155790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.156422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:05.156728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.158217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.164086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 471/941] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:05.169556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.262417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.285274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3056220-6ba6-4479-af99-dd2672e14d90 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.320250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.320854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.321133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.321404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.322024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:05.322320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.323701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.328910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 471/942] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:05.335350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.446918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630c6f8-da67-41f4-9bba-078e08a28d51 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.485182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63476c8e-f6cf-4ab1-915e-80aac7051f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.486177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63476c8e-f6cf-4ab1-915e-80aac7051f7a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.486692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63476c8e-f6cf-4ab1-915e-80aac7051f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.487184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63476c8e-f6cf-4ab1-915e-80aac7051f7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.488127 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63476c8e-f6cf-4ab1-915e-80aac7051f7a None admin] RBAC: Adding query filter params (interface=public, service_id=5b0ba4dfb08341ec903fa04f60d61b22, region_id=RegionOne) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:05.489230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63476c8e-f6cf-4ab1-915e-80aac7051f7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.490803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63476c8e-f6cf-4ab1-915e-80aac7051f7a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.496589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 472/943] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/endpoints?service_id=5b0ba4dfb08341ec903fa04f60d61b22&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:05.502332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.607333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed12f1-b6aa-4e6f-acd9-35fa79d3cd00 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.641794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee4357-e770-4f99-b140-a2c2e30f3492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.642746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee4357-e770-4f99-b140-a2c2e30f3492 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.643249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee4357-e770-4f99-b140-a2c2e30f3492 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.643712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee4357-e770-4f99-b140-a2c2e30f3492 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.644491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee4357-e770-4f99-b140-a2c2e30f3492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.646049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee4357-e770-4f99-b140-a2c2e30f3492 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.652507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 472/944] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:47:06.536953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec06a5-77b7-430a-b1da-3c4bb754d3fa None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.537472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec06a5-77b7-430a-b1da-3c4bb754d3fa None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.537472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec06a5-77b7-430a-b1da-3c4bb754d3fa None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.539181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 473/945] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:06.545299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e535-4675-482c-8b37-a44d4a45ebaf None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.545805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e535-4675-482c-8b37-a44d4a45ebaf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.545905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e535-4675-482c-8b37-a44d4a45ebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.559790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:06.565809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5a39e535-4675-482c-8b37-a44d4a45ebaf None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:06.567702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5a39e535-4675-482c-8b37-a44d4a45ebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:06.645183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 473/946] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:06.742383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378760-e31b-42a4-b71d-09798e5421ff None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.742753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378760-e31b-42a4-b71d-09798e5421ff None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.743170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378760-e31b-42a4-b71d-09798e5421ff None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.744587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 474/947] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:06.749790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6291f69-a98d-43ad-93b9-f9083462b421 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.750187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6291f69-a98d-43ad-93b9-f9083462b421 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.750570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6291f69-a98d-43ad-93b9-f9083462b421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.769400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d6291f69-a98d-43ad-93b9-f9083462b421 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:06.770616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d6291f69-a98d-43ad-93b9-f9083462b421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:06.779401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c7ee9-8e11-4b0e-9329-dd39bbc62d4a None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.779695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c7ee9-8e11-4b0e-9329-dd39bbc62d4a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.779975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c7ee9-8e11-4b0e-9329-dd39bbc62d4a None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.781537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 475/948] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:06.786369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811859b-8e3a-40a5-aa7e-b8139fdf03cd None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.786656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811859b-8e3a-40a5-aa7e-b8139fdf03cd None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.786963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811859b-8e3a-40a5-aa7e-b8139fdf03cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.805676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c811859b-8e3a-40a5-aa7e-b8139fdf03cd None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:06.806871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c811859b-8e3a-40a5-aa7e-b8139fdf03cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:06.830342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:06.845015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 474/949] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:06.878783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 476/950] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:06.917852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6291f69-a98d-43ad-93b9-f9083462b421 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.983030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.018352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6291f69-a98d-43ad-93b9-f9083462b421 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.052615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.053222 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.053501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.053770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.054403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.055853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.058434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING keystone.server.flask.common [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 10 11:47:07.080198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.sql.core [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 11:47:07.080198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 11:47:07.080198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [parameters: {'id': '822d3f6ebc3e451ebb36a6214c774f96', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 10 11:47:07.080198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81458) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 822d3f6ebc3e451ebb36a6214c774f96.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 822d3f6ebc3e451ebb36a6214c774f96. Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:47:07.096552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result.read() Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 10 11:47:07.097348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application yield resource Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise self.value Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application session.commit() Oct 10 11:47:07.098838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.session.flush() Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application compat.raise_( Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 11:47:07.099763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 11:47:07.100648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result.read() Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:47:07.101422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [parameters: {'id': '822d3f6ebc3e451ebb36a6214c774f96', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.102276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:07.107281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 822d3f6ebc3e451ebb36a6214c774f96. Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.109365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 475/951] 10.222.0.52 () {60 vars in 1140 bytes} [Mon Oct 10 11:47:06 2022] POST /identity/v3/projects => generated 262 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:07.110609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811859b-8e3a-40a5-aa7e-b8139fdf03cd None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.158134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.215395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811859b-8e3a-40a5-aa7e-b8139fdf03cd None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.250937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.251570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.251853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.252126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.258675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.260141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.270512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdfb54-1124-46b6-b0ad-e791bcfab4ff None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:07.271589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:07.272438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 477/952] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:07.273940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.298072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.300409 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.315927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.316592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.316876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.317151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.317739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.319340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:07.328446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:07.334234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.335814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 476/953] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:07.336688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.391889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624022e2-991c-4e35-be35-5bb4b8513e55 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.437750 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f67a82-3b5a-4c8e-b019-6ec10d99ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.438381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67a82-3b5a-4c8e-b019-6ec10d99ade5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.438661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67a82-3b5a-4c8e-b019-6ec10d99ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.438956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67a82-3b5a-4c8e-b019-6ec10d99ade5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.442346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0527ab3-8198-443e-b82d-9d39098fccbb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.451126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f67a82-3b5a-4c8e-b019-6ec10d99ade5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-a3f67a82-3b5a-4c8e-b019-6ec10d99ade5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:07.457115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:07.458131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:07.458991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 478/954] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:07.459778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f67a82-3b5a-4c8e-b019-6ec10d99ade5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.484668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.485273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.485554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.485898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.486476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:07.486774 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.488947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.509796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 477/955] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:07.510622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.563758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.566307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.602223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f67a82-3b5a-4c8e-b019-6ec10d99ade5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.628955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64607e-69d2-44d1-9766-aea6345e9be5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.652235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.654002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.654855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.655343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.656487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:07.657474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.659383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.672280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 479/956] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:07.673820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.675820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.677085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.677402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.677682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.678360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:07.678735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.680821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.694917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 478/957] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:07.696994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.819200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9466d-45ca-44fa-aa1e-13a2631a01b6 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.824302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664c603-4bae-4238-85f9-797859e95294 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.836145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.861234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725ca02-b7d3-48d1-a317-4784bc8357cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.861433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69130-4b3c-4897-ac9b-cf2ac6b64186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.861846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69130-4b3c-4897-ac9b-cf2ac6b64186 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.862124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69130-4b3c-4897-ac9b-cf2ac6b64186 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.862397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69130-4b3c-4897-ac9b-cf2ac6b64186 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.862922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ca02-b7d3-48d1-a317-4784bc8357cb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.863273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c69130-4b3c-4897-ac9b-cf2ac6b64186 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:07.864138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c69130-4b3c-4897-ac9b-cf2ac6b64186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.864319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ca02-b7d3-48d1-a317-4784bc8357cb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.864959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ca02-b7d3-48d1-a317-4784bc8357cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.865501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c69130-4b3c-4897-ac9b-cf2ac6b64186 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.875081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 480/958] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:07.879246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725ca02-b7d3-48d1-a317-4784bc8357cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.881051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725ca02-b7d3-48d1-a317-4784bc8357cb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.897539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c871a-2c2c-4c3b-ae30-cbb590ea76eb None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.897887 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c871a-2c2c-4c3b-ae30-cbb590ea76eb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.898085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c871a-2c2c-4c3b-ae30-cbb590ea76eb None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.901847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 481/959] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9725ca02-b7d3-48d1-a317-4784bc8357cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:07.902130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:07.904308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:07.906398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:07.906398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:07.906398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:07.906398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 479/960] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:07.911657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c871a-2c2c-4c3b-ae30-cbb590ea76eb None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.913877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb330-be1e-4788-b0a8-819195903f0b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.914164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb330-be1e-4788-b0a8-819195903f0b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.914442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb330-be1e-4788-b0a8-819195903f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.918921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.939507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9eecb330-be1e-4788-b0a8-819195903f0b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:07.941135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9eecb330-be1e-4788-b0a8-819195903f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:08.037812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 480/961] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:08.052139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c871a-2c2c-4c3b-ae30-cbb590ea76eb None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.089452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.090070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.090353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.090627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.091303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:08.091608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.092972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.105366 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 482/962] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:08.110946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb330-be1e-4788-b0a8-819195903f0b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.205502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.216188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb330-be1e-4788-b0a8-819195903f0b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.252632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.253269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.253553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.253824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.261209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.262644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:08.274768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:08.275740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:08.276672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 481/963] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:08.281858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.326447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.355586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.415020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d6011-3487-443b-9d6f-fb01eaacca02 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.448583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88ac6f-d0c6-42ab-b6a8-e5b06d7dbed8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.451533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.452217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.452567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.452952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.453745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:08.454116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.455666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.476165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 483/964] 10.222.0.52 () {56 vars in 1149 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:08.481586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.495731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.496668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.497298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.498165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.499239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.501082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:08.513031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:08.514279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:08.515340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:08.518486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 482/965] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:08.524068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.563314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.603196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc4761-4f10-4da4-9a4a-8ca134f3822e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.627301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.640042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79c0f-efa7-41a6-9f8a-886bed798c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.640385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79c0f-efa7-41a6-9f8a-886bed798c9e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.640672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79c0f-efa7-41a6-9f8a-886bed798c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.640948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79c0f-efa7-41a6-9f8a-886bed798c9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.656305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7852d-12c5-40ab-b836-4e035f2f2365 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.700490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.701191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.701544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.701884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.702584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:08.703004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.704441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.714654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 483/966] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:08.716048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.723557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f79c0f-efa7-41a6-9f8a-886bed798c9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.725797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f79c0f-efa7-41a6-9f8a-886bed798c9e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.799136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.822870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b2db5-aee8-4a2a-a0cf-36692c75d3b1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.844282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 484/967] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 11:47:08.849631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79c0f-efa7-41a6-9f8a-886bed798c9e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.867571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.868542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.869029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.869461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.870233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:08.870680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.872429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.883069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 484/968] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:08.883221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.883874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.974093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79c0f-efa7-41a6-9f8a-886bed798c9e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.995320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e48a4-39a9-4c0b-9cbf-f66781427c40 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.011038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.011769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.012054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.012325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.023469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.025016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.033942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7e321-424d-4bfb-b1df-bdbeb74634b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.034561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7e321-424d-4bfb-b1df-bdbeb74634b6 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.034841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7e321-424d-4bfb-b1df-bdbeb74634b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.035142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7e321-424d-4bfb-b1df-bdbeb74634b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.036363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7e321-424d-4bfb-b1df-bdbeb74634b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.037876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7e321-424d-4bfb-b1df-bdbeb74634b6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:09.040537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:09.041441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:09.042308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:09.042308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:09.042308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:09.042308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 485/969] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:09.047264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.064254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.073269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 485/970] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:08 2022] POST /identity/v3/endpoints => generated 349 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:09.150327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.166524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c93421-01a2-4f35-b782-5305975dfe3c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.210551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.211205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.211484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.211753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.212400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:09.212702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.214107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.227862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 486/971] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:09.236231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7e321-424d-4bfb-b1df-bdbeb74634b6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.350240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7e321-424d-4bfb-b1df-bdbeb74634b6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.385492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.386385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.386815 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.387309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.394242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.396422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:09.408355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:09.409513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:09.410644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:09.410644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:09.410644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:09.410644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:09.410644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:09.410644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:09.410644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:09.410644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:09.414592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 486/972] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:09.422485 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.526474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adb677-53a1-4242-b4ee-82614222d83c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.556421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.562814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2b887-c197-4912-a3c7-ba1e8bba45a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.563450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2b887-c197-4912-a3c7-ba1e8bba45a0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.563730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2b887-c197-4912-a3c7-ba1e8bba45a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.564038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2b887-c197-4912-a3c7-ba1e8bba45a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.564699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2b887-c197-4912-a3c7-ba1e8bba45a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:09.565012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2b887-c197-4912-a3c7-ba1e8bba45a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.566464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2b887-c197-4912-a3c7-ba1e8bba45a0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.579386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 487/973] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:09.585269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.652429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.698672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa4d4-beeb-476d-8b22-3ad4d2a3d919 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.742269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6f3fe-11f1-4938-a5f5-f0f805a51fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.742917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f3fe-11f1-4938-a5f5-f0f805a51fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.743212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f3fe-11f1-4938-a5f5-f0f805a51fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.743479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f3fe-11f1-4938-a5f5-f0f805a51fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.744235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6f3fe-11f1-4938-a5f5-f0f805a51fa1 None admin] RBAC: Adding query filter params (role.id=2b49ddcd3d3346a4879bbf33c27d9419, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=cf591850b9914f2f9089c8b6df730b4a) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:09.744543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6f3fe-11f1-4938-a5f5-f0f805a51fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.746038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6f3fe-11f1-4938-a5f5-f0f805a51fa1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.753170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 487/974] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/role_assignments?user.id=cf591850b9914f2f9089c8b6df730b4a&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2b49ddcd3d3346a4879bbf33c27d9419 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:09.870623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fa447-1add-46ed-871d-e04e3420e448 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.871197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fa447-1add-46ed-871d-e04e3420e448 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.871197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fa447-1add-46ed-871d-e04e3420e448 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.872862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 488/975] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:09.878741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8435ce-0995-4a86-a442-3811762a8c20 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.879639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8435ce-0995-4a86-a442-3811762a8c20 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.879906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8435ce-0995-4a86-a442-3811762a8c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.899434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9c8435ce-0995-4a86-a442-3811762a8c20 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:09.900677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9c8435ce-0995-4a86-a442-3811762a8c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:09.976076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 488/976] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:10.145036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581005c-c81f-4c6d-8af0-10f7149c55b4 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.145429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581005c-c81f-4c6d-8af0-10f7149c55b4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.145611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581005c-c81f-4c6d-8af0-10f7149c55b4 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.147066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 489/977] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:10.152424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24c80-dc5d-4f27-96a9-21bd9158c748 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.152720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24c80-dc5d-4f27-96a9-21bd9158c748 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.152996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24c80-dc5d-4f27-96a9-21bd9158c748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.171974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f0a24c80-dc5d-4f27-96a9-21bd9158c748 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:10.173148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f0a24c80-dc5d-4f27-96a9-21bd9158c748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:10.222307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:10.247144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 489/978] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:10.258998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581005c-c81f-4c6d-8af0-10f7149c55b4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.366252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581005c-c81f-4c6d-8af0-10f7149c55b4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.401911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.402517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.402798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.403097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.406232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.406232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.407871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING keystone.server.flask.common [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 10 11:47:10.422133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.sql.core [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 11:47:10.422133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 10 11:47:10.422133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [parameters: {'user_id': '3959154b6d104dbfaa26edb892b39b4a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 10 11:47:10.422133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81457) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result.read() Oct 10 11:47:10.429104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:47:10.429980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application yield resource Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise self.value Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application session.commit() Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 11:47:10.430965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.session.flush() Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application compat.raise_( Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 11:47:10.431869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:47:10.432774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application result.read() Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:47:10.433646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [parameters: {'user_id': '3959154b6d104dbfaa26edb892b39b4a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:10.434476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 10 11:47:10.436775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 10 11:47:10.439360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 10 11:47:10.439360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 10 11:47:10.439360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 11:47:10.439360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 11:47:10.439360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 10 11:47:10.439360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:10.439360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 490/979] 10.222.0.52 () {60 vars in 1131 bytes} [Mon Oct 10 11:47:10 2022] POST /identity/v3/users => generated 160 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:10.440247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24c80-dc5d-4f27-96a9-21bd9158c748 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.525280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.553320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24c80-dc5d-4f27-96a9-21bd9158c748 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.583884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:10.602126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.602782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.603095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.603370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.612339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.614475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:10.631085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:10.632137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:10.633070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 490/980] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:10.639796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.640390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d9924-25c1-4670-9de5-5367413488d8 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.685839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc09897-54e2-4da5-bbf7-89665f0f4a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.686445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc09897-54e2-4da5-bbf7-89665f0f4a82 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.686724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc09897-54e2-4da5-bbf7-89665f0f4a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.687023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc09897-54e2-4da5-bbf7-89665f0f4a82 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.688387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc09897-54e2-4da5-bbf7-89665f0f4a82 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.690192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:10.691380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc09897-54e2-4da5-bbf7-89665f0f4a82 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.723189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 491/981] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:10 2022] POST /identity/v3/services => generated 242 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:10.765920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decaf0a-8b45-4351-ab31-c0e47aa506ea None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.813730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58db12b-3c7a-476e-a612-b7a013d33290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.814266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db12b-3c7a-476e-a612-b7a013d33290 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.814543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db12b-3c7a-476e-a612-b7a013d33290 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.814812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db12b-3c7a-476e-a612-b7a013d33290 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.815557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58db12b-3c7a-476e-a612-b7a013d33290 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:10.817208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58db12b-3c7a-476e-a612-b7a013d33290 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.817997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58db12b-3c7a-476e-a612-b7a013d33290 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.858076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 491/982] 10.222.0.52 () {56 vars in 1101 bytes} [Mon Oct 10 11:47:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:11.411117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae9e1-790d-43a1-8e50-ad627537a29c None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:11.411567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae9e1-790d-43a1-8e50-ad627537a29c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:11.411644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dae9e1-790d-43a1-8e50-ad627537a29c None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:11.414660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 492/983] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:11.418727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d356f-01fa-4ce7-ad5c-9d98c3531e86 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:11.419066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d356f-01fa-4ce7-ad5c-9d98c3531e86 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:11.419344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d356f-01fa-4ce7-ad5c-9d98c3531e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:11.438243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1f0d356f-01fa-4ce7-ad5c-9d98c3531e86 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:11.439463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1f0d356f-01fa-4ce7-ad5c-9d98c3531e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:11.508220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 492/984] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:11.780712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dae9e1-790d-43a1-8e50-ad627537a29c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:11.881145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dae9e1-790d-43a1-8e50-ad627537a29c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:11.894171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:11.916833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b5a46-0128-48ff-85fa-c6cbd948a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:11.917496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b5a46-0128-48ff-85fa-c6cbd948a8ec None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:11.917773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b5a46-0128-48ff-85fa-c6cbd948a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:11.918044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b5a46-0128-48ff-85fa-c6cbd948a8ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:11.918632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b5a46-0128-48ff-85fa-c6cbd948a8ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:11.920082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b5a46-0128-48ff-85fa-c6cbd948a8ec None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-f98b5a46-0128-48ff-85fa-c6cbd948a8ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:11.930353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:11.931479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:11.932390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:11.935282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 493/985] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:11.938079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d356f-01fa-4ce7-ad5c-9d98c3531e86 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:11.966689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:12.040631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204ebb8-37e4-44d4-a022-b0b54ddbfef3 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.040836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204ebb8-37e4-44d4-a022-b0b54ddbfef3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.041084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204ebb8-37e4-44d4-a022-b0b54ddbfef3 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.042687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 494/986] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:12.043597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d356f-01fa-4ce7-ad5c-9d98c3531e86 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.047576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836848-7c84-48c4-a94e-5977f9edba68 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.047861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836848-7c84-48c4-a94e-5977f9edba68 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.048132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836848-7c84-48c4-a94e-5977f9edba68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.066238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-43836848-7c84-48c4-a94e-5977f9edba68 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:12.067808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-43836848-7c84-48c4-a94e-5977f9edba68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:12.080181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.080791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.081069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.081341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.082475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:12.082789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.084169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.089087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 493/987] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:12.094585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.143258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 495/988] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:12.189935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:12.200501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a44da4-f133-4f70-9a13-659bf5270746 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.234108 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.234706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.235016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.235286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.235900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:12.236196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.237570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.242396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 494/989] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:12.248194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836848-7c84-48c4-a94e-5977f9edba68 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.346463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836848-7c84-48c4-a94e-5977f9edba68 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.353742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:12.381152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.381763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.382042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.382312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.383048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] RBAC: Adding query filter params (interface=admin, service_id=5b0ba4dfb08341ec903fa04f60d61b22, region_id=RegionOne) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:12.383357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.384711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.389943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 496/990] 10.222.0.52 () {56 vars in 1249 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/endpoints?service_id=5b0ba4dfb08341ec903fa04f60d61b22&interface=admin®ion_id=RegionOne => generated 193 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:12.396129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.413307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.510739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3db8b-20cc-49c9-8ead-014f6b5640a8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.518835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162edf2a-6ea3-4877-bd8a-e5ae9329c2c4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.547086 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b51bc-31f5-4d5e-bb76-2110958f1e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.547693 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b51bc-31f5-4d5e-bb76-2110958f1e12 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.548134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b51bc-31f5-4d5e-bb76-2110958f1e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.548408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b51bc-31f5-4d5e-bb76-2110958f1e12 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.549356 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b51bc-31f5-4d5e-bb76-2110958f1e12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.551255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b51bc-31f5-4d5e-bb76-2110958f1e12 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.554018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.554648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.554950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.555225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.555810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.557152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.567151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 495/991] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:12.576291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:12.577380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:12.583351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 497/992] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:12.585750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b51bc-31f5-4d5e-bb76-2110958f1e12 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.606666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:12.697565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b51bc-31f5-4d5e-bb76-2110958f1e12 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.735960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ac933-721a-4524-a759-f9da937a922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.736571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac933-721a-4524-a759-f9da937a922e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.736850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac933-721a-4524-a759-f9da937a922e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.737119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac933-721a-4524-a759-f9da937a922e None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.737749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ac933-721a-4524-a759-f9da937a922e None admin] RBAC: Adding query filter params (name=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:12.738053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ac933-721a-4524-a759-f9da937a922e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.740032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ac933-721a-4524-a759-f9da937a922e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.745464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 496/993] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:12.751150 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.789371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:12.862668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748c5d8-aad9-400f-8143-d96ab140b5f9 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.898164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c536e42-a97d-4097-8e47-233bd28060e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.898769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536e42-a97d-4097-8e47-233bd28060e3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.899071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536e42-a97d-4097-8e47-233bd28060e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.899337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536e42-a97d-4097-8e47-233bd28060e3 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.899960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c536e42-a97d-4097-8e47-233bd28060e3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:12.900258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c536e42-a97d-4097-8e47-233bd28060e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.901668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c536e42-a97d-4097-8e47-233bd28060e3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.906443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 498/994] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:12.911561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc1274-c03c-43b0-9df4-61ab0e82471b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.911841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc1274-c03c-43b0-9df4-61ab0e82471b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.912110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc1274-c03c-43b0-9df4-61ab0e82471b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.913649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 497/995] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:12.919367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc2976-4a2c-4ee0-a148-c3b3ece1be6d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.919688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc2976-4a2c-4ee0-a148-c3b3ece1be6d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.919928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc2976-4a2c-4ee0-a148-c3b3ece1be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.940023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-2cbc2976-4a2c-4ee0-a148-c3b3ece1be6d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:12.941267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2cbc2976-4a2c-4ee0-a148-c3b3ece1be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:13.016037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 499/996] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:13.284524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc1274-c03c-43b0-9df4-61ab0e82471b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:13.369076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:13.389923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc1274-c03c-43b0-9df4-61ab0e82471b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:13.428965 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:13.429643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.429924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.430191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.430765 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:13.432210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:13.442433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:13.443547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:13.444400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 498/997] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:13.449790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc2976-4a2c-4ee0-a148-c3b3ece1be6d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:13.478607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:13.552490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc2976-4a2c-4ee0-a148-c3b3ece1be6d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:13.586034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:13.586645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.586951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.587228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.587856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:13.588164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:13.589524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:13.594501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 500/998] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:13.600011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:13.700996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da67a8-3d08-48cd-aac7-3c2089f91ec1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:13.734269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:13.734873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.735180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.735453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.736075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:13.736373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:13.737722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:13.742601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 499/999] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:13.751429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:13.851062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:13.865319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3b87e-cf37-4feb-872a-ec118b68caa7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:13.899684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db208827-ef1b-4353-9ad5-17926b83bd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:13.900293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db208827-ef1b-4353-9ad5-17926b83bd7b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.900575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db208827-ef1b-4353-9ad5-17926b83bd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.900897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db208827-ef1b-4353-9ad5-17926b83bd7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.901596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db208827-ef1b-4353-9ad5-17926b83bd7b None admin] RBAC: Adding query filter params (interface=public, service_id=55fca79186a7481e9bde060af350a965, region_id=RegionOne) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:13.901904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db208827-ef1b-4353-9ad5-17926b83bd7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:13.903294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db208827-ef1b-4353-9ad5-17926b83bd7b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:13.908455 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 501/1000] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:13 2022] GET /identity/v3/endpoints?service_id=55fca79186a7481e9bde060af350a965&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:13.913921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:13.926625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:14.016640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd197b53-8797-436a-a787-292dad3161c0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:14.049082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02158f7-1a51-4fb3-8b04-234b8746c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:14.049686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02158f7-1a51-4fb3-8b04-234b8746c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.049964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02158f7-1a51-4fb3-8b04-234b8746c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.050235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02158f7-1a51-4fb3-8b04-234b8746c8d3 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.051384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02158f7-1a51-4fb3-8b04-234b8746c8d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:14.052730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02158f7-1a51-4fb3-8b04-234b8746c8d3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:14.058930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 500/1001] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:13 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:47:14.734581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31248d-a9ac-411b-87a9-0955839b0f23 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.735123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31248d-a9ac-411b-87a9-0955839b0f23 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.735123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31248d-a9ac-411b-87a9-0955839b0f23 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.736571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 502/1002] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:14.742094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb60a5-0005-43ec-954c-1fa741b01946 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.742502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb60a5-0005-43ec-954c-1fa741b01946 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.742677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb60a5-0005-43ec-954c-1fa741b01946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.761335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-31cb60a5-0005-43ec-954c-1fa741b01946 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:14.762504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-31cb60a5-0005-43ec-954c-1fa741b01946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:14.829242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 501/1003] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:15.072195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe3169-7c81-4fde-865f-2db266c90372 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.072525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe3169-7c81-4fde-865f-2db266c90372 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.072740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe3169-7c81-4fde-865f-2db266c90372 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.074235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 503/1004] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:15.079377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7864a28-4a59-4966-aa22-9341d4c0163b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.079676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7864a28-4a59-4966-aa22-9341d4c0163b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.079955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7864a28-4a59-4966-aa22-9341d4c0163b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.087701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:15.099534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe3169-7c81-4fde-865f-2db266c90372 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:15.099684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e7864a28-4a59-4966-aa22-9341d4c0163b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:15.100809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e7864a28-4a59-4966-aa22-9341d4c0163b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:15.175026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 502/1005] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:15.215869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe3169-7c81-4fde-865f-2db266c90372 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:15.249659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:15.250326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.250609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.250881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.251484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:15.252918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:15.262178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:15.266998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:15.268219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 504/1006] 10.222.0.52 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:15.270364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7864a28-4a59-4966-aa22-9341d4c0163b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:15.371799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7864a28-4a59-4966-aa22-9341d4c0163b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:15.411218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:15.414258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.414258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.414258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.414258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:15.414258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:15.414604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:15.417250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:15.422099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 503/1007] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:15.428448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:15.440662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:15.452557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:15.542368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf7054-468a-4946-a896-a84161eb1cb9 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:15.554114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb435a-cd5f-43c4-94ca-0b3fda90efdd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:15.579541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:15.580165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.580406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.580671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.581288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:15.581587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:15.583493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:15.593121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 505/1008] 10.222.0.52 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:15.593276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffe2ea-86a1-4ba9-8f4e-311ad9429fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:15.593276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffe2ea-86a1-4ba9-8f4e-311ad9429fe6 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.593276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffe2ea-86a1-4ba9-8f4e-311ad9429fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.593276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffe2ea-86a1-4ba9-8f4e-311ad9429fe6 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.593557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffe2ea-86a1-4ba9-8f4e-311ad9429fe6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:15.593557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffe2ea-86a1-4ba9-8f4e-311ad9429fe6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:15.594143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:15.610698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 504/1009] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:15 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:47:15.695184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:15.713024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3329c3-3c76-404b-b214-8bcaa0a61569 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:15.755897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1118f19-d3ea-4aec-92a7-725d0fd3ac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:15.756535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1118f19-d3ea-4aec-92a7-725d0fd3ac09 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.756790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1118f19-d3ea-4aec-92a7-725d0fd3ac09 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.757030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1118f19-d3ea-4aec-92a7-725d0fd3ac09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.757693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1118f19-d3ea-4aec-92a7-725d0fd3ac09 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:15.759091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1118f19-d3ea-4aec-92a7-725d0fd3ac09 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:15.783154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 506/1010] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:16.258663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0bfc-7dc0-402f-8b7f-f350f283ce9d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:16.259203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0bfc-7dc0-402f-8b7f-f350f283ce9d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:16.259282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0bfc-7dc0-402f-8b7f-f350f283ce9d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:16.260752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 505/1011] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:16.265735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8643-2b50-4b69-a524-381255e9b7ee None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:16.266031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8643-2b50-4b69-a524-381255e9b7ee None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:16.266303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8643-2b50-4b69-a524-381255e9b7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:16.285557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3b8f8643-2b50-4b69-a524-381255e9b7ee None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:16.286760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3b8f8643-2b50-4b69-a524-381255e9b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:16.360370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 507/1012] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:16 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:16.631088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c0bfc-7dc0-402f-8b7f-f350f283ce9d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:16.693689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:16.737717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c0bfc-7dc0-402f-8b7f-f350f283ce9d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:16.771297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:16.771926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:16.772206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:16.772481 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] PATH_INFO: `/v3/services/compute` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:16.773055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:16.774406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:16.783143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:16.784083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:16.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:16.787245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 506/1013] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:16.790119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f8643-2b50-4b69-a524-381255e9b7ee None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:16.858173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:16.895086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f8643-2b50-4b69-a524-381255e9b7ee None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:16.928999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:16.929612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:16.929893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:16.930163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:16.930846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:16.931183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:16.932553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:16.942833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 508/1014] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:16.944057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:17.048992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239aa507-94e3-4ac3-a5e2-48696aad3eea None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:17.074463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:17.085566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ffa7-3659-4344-b0c5-300fd15bac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:17.086172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ffa7-3659-4344-b0c5-300fd15bac3f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.086450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ffa7-3659-4344-b0c5-300fd15bac3f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.086720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ffa7-3659-4344-b0c5-300fd15bac3f None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.087375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ffa7-3659-4344-b0c5-300fd15bac3f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:17.087676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ffa7-3659-4344-b0c5-300fd15bac3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:17.089025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ffa7-3659-4344-b0c5-300fd15bac3f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:17.098300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 507/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:17.099511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:17.202075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b7959-0e5e-4a91-a779-f3fbf3fd9c2e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:17.230469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:17.236998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50d0c9-5310-485f-ae10-7c38f0af6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:17.237606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d0c9-5310-485f-ae10-7c38f0af6876 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.237881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d0c9-5310-485f-ae10-7c38f0af6876 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.238661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50d0c9-5310-485f-ae10-7c38f0af6876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.239347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50d0c9-5310-485f-ae10-7c38f0af6876 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:17.240699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50d0c9-5310-485f-ae10-7c38f0af6876 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:17.266935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 509/1016] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:17.819312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b37e-4236-4dad-b49c-29a64ce548a8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.819852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b37e-4236-4dad-b49c-29a64ce548a8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.819852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b37e-4236-4dad-b49c-29a64ce548a8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.821442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 508/1017] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:17.827190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61791b-d742-481b-99b9-5a62e5771df4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.827703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61791b-d742-481b-99b9-5a62e5771df4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.827778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61791b-d742-481b-99b9-5a62e5771df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.848537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-7c61791b-d742-481b-99b9-5a62e5771df4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:17.849762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7c61791b-d742-481b-99b9-5a62e5771df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:17.924016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 510/1018] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:17.944500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eff38-d63a-468d-a646-4377d965790d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.944784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eff38-d63a-468d-a646-4377d965790d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.945053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eff38-d63a-468d-a646-4377d965790d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.946498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 509/1019] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:17.951486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff590980-d1eb-4b30-a802-fb9e647bf74a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.951775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff590980-d1eb-4b30-a802-fb9e647bf74a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.952048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff590980-d1eb-4b30-a802-fb9e647bf74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.970660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ff590980-d1eb-4b30-a802-fb9e647bf74a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:17.971856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ff590980-d1eb-4b30-a802-fb9e647bf74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:18.037660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 511/1020] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:18.191895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271eff38-d63a-468d-a646-4377d965790d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.295911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271eff38-d63a-468d-a646-4377d965790d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.308303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff590980-d1eb-4b30-a802-fb9e647bf74a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.331438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.332784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.333420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.333700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.334430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.335829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:18.344567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:18.345602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:18.347916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 510/1021] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:18.351405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.352022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:18.407931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:18.418693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff590980-d1eb-4b30-a802-fb9e647bf74a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.459662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53663-64f2-4fa3-b491-fa562e5ea2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.460214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53663-64f2-4fa3-b491-fa562e5ea2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.460496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53663-64f2-4fa3-b491-fa562e5ea2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.460766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53663-64f2-4fa3-b491-fa562e5ea2d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.461381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e72ac7-a760-49b8-87f2-ccc29db607d1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.461880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53663-64f2-4fa3-b491-fa562e5ea2d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.464473 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53663-64f2-4fa3-b491-fa562e5ea2d1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.466238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 512/1022] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:18.471656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53663-64f2-4fa3-b491-fa562e5ea2d1 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.502417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.503057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.503601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.503601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.504251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:18.504550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.505984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.515932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 511/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:18.516926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.581510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53663-64f2-4fa3-b491-fa562e5ea2d1 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.592689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:18.618738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706f607-5ca0-4372-98f4-d9eecabacb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.619369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706f607-5ca0-4372-98f4-d9eecabacb66 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.619643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706f607-5ca0-4372-98f4-d9eecabacb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.619872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde19d41-07d3-4323-9c1d-d13ad102aba9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.619987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706f607-5ca0-4372-98f4-d9eecabacb66 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.620530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706f607-5ca0-4372-98f4-d9eecabacb66 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.621942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706f607-5ca0-4372-98f4-d9eecabacb66 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.635080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:18.658155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.658783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.659090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.659362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.660563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] RBAC: Adding query filter params (type=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:18.660861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.662253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.663266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 513/1024] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:18 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:18.673269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 512/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:18.674429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706f607-5ca0-4372-98f4-d9eecabacb66 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.784561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706f607-5ca0-4372-98f4-d9eecabacb66 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.826649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d8d51-2c5a-44f0-b7d5-3255189bffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.827283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d8d51-2c5a-44f0-b7d5-3255189bffef None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.827796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d8d51-2c5a-44f0-b7d5-3255189bffef None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.827796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d8d51-2c5a-44f0-b7d5-3255189bffef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.828825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d8d51-2c5a-44f0-b7d5-3255189bffef None admin] RBAC: Adding query filter params (interface=public, service_id=8f25c9c0ebdc46189541f79b15f7e6a1, region_id=RegionOne) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:18.828825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d8d51-2c5a-44f0-b7d5-3255189bffef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.830390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d8d51-2c5a-44f0-b7d5-3255189bffef None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.835499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 514/1026] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/endpoints?service_id=8f25c9c0ebdc46189541f79b15f7e6a1&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:18.841080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.942822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830a965-63ca-4c3a-909d-0387a3a4c6ff None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.976281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b05e4-cd33-4f5e-9b6b-ef43ca085c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.976883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b05e4-cd33-4f5e-9b6b-ef43ca085c02 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.977158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b05e4-cd33-4f5e-9b6b-ef43ca085c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.977422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b05e4-cd33-4f5e-9b6b-ef43ca085c02 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.978012 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b05e4-cd33-4f5e-9b6b-ef43ca085c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.979394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b05e4-cd33-4f5e-9b6b-ef43ca085c02 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.986045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 513/1027] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:47:19.526359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e5004-d19b-44d0-83e9-77959d5f6f58 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:19.526668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e5004-d19b-44d0-83e9-77959d5f6f58 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:19.526943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e5004-d19b-44d0-83e9-77959d5f6f58 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:19.528445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 515/1028] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:19.533412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057354-089e-4477-ae8e-5a9af6cfa699 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:19.533741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057354-089e-4477-ae8e-5a9af6cfa699 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:19.533990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057354-089e-4477-ae8e-5a9af6cfa699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:19.552442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c1057354-089e-4477-ae8e-5a9af6cfa699 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:19.553635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c1057354-089e-4477-ae8e-5a9af6cfa699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:19.592397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:19.625127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 514/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:19.890978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e5004-d19b-44d0-83e9-77959d5f6f58 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:19.919161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:19.996465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e5004-d19b-44d0-83e9-77959d5f6f58 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.029987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54f28a-eeaf-4637-8475-b7064afa489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.030611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54f28a-eeaf-4637-8475-b7064afa489f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.030905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54f28a-eeaf-4637-8475-b7064afa489f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.031186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54f28a-eeaf-4637-8475-b7064afa489f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.041614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54f28a-eeaf-4637-8475-b7064afa489f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-0f54f28a-eeaf-4637-8475-b7064afa489f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:20.046323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:20.047296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:20.048171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:20.051051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 516/1030] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Oct 10 11:47:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:20.053337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1057354-089e-4477-ae8e-5a9af6cfa699 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.152153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1057354-089e-4477-ae8e-5a9af6cfa699 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.185614 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.186221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.186500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.186771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.187722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:20.188022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.189375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.201817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 515/1031] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:20.203102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54f28a-eeaf-4637-8475-b7064afa489f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.292005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:20.308067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54f28a-eeaf-4637-8475-b7064afa489f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.342791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.343465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.343738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.344004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.354573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.356040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:20.370236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:20.371239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:20.372146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:20.372146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:20.372146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:20.375226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 517/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:20.376944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.418268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:20.488228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fed5cc-f9f3-4208-be06-8e1fde5c9e16 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.522064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.522666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.522965 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.523238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.523871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:20.524168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.525506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.537947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 516/1033] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:20.543335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.647469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bbdad-2a68-4318-935c-fe059536cc2f None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.682247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046fd7c-a655-496c-b1ea-ddf5f153831b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.682804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046fd7c-a655-496c-b1ea-ddf5f153831b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.683147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046fd7c-a655-496c-b1ea-ddf5f153831b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.683423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046fd7c-a655-496c-b1ea-ddf5f153831b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.691849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046fd7c-a655-496c-b1ea-ddf5f153831b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.693284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046fd7c-a655-496c-b1ea-ddf5f153831b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.695059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-9046fd7c-a655-496c-b1ea-ddf5f153831b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:20.705749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:20.706800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:20.708583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:20.709572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 518/1034] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:20.713185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.815492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:20.824783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4c6a2-64f3-4f02-97f4-866e98e17353 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.853427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64da9d-90c5-482f-915a-c94668086490 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.853781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64da9d-90c5-482f-915a-c94668086490 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.853983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64da9d-90c5-482f-915a-c94668086490 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.855480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 519/1035] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:20.860267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608d9a7-7802-4936-80ce-374ce0b74963 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.860556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608d9a7-7802-4936-80ce-374ce0b74963 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.860832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608d9a7-7802-4936-80ce-374ce0b74963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.862749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.863982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.864196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.864558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.865374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:20.865673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.867199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.880797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d608d9a7-7802-4936-80ce-374ce0b74963 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:20.881991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d608d9a7-7802-4936-80ce-374ce0b74963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:20.886751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 517/1036] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:20.892091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.951568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 520/1037] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:21.001112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4626f-b470-4a6d-a9df-fb6e6719acce None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.040327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3eaef0-6c00-4ba8-bb06-8356a4ca3fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.040943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eaef0-6c00-4ba8-bb06-8356a4ca3fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.041223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eaef0-6c00-4ba8-bb06-8356a4ca3fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.041495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eaef0-6c00-4ba8-bb06-8356a4ca3fc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.063339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.112945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3eaef0-6c00-4ba8-bb06-8356a4ca3fc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.114387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3eaef0-6c00-4ba8-bb06-8356a4ca3fc4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.128409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ad21-56b0-4493-b4ae-da40323dbff8 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.129227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ad21-56b0-4493-b4ae-da40323dbff8 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.129495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ad21-56b0-4493-b4ae-da40323dbff8 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.130816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 521/1038] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:21.135640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e28d6-7ddb-4929-9fc7-c41f4cf3f8ee None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.135919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e28d6-7ddb-4929-9fc7-c41f4cf3f8ee None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.136190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e28d6-7ddb-4929-9fc7-c41f4cf3f8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.154310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5c9e28d6-7ddb-4929-9fc7-c41f4cf3f8ee None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:21.155494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5c9e28d6-7ddb-4929-9fc7-c41f4cf3f8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:21.217066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 518/1039] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 11:47:21.222845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3eaef0-6c00-4ba8-bb06-8356a4ca3fc4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.228332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 522/1040] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:21.229207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e28d6-7ddb-4929-9fc7-c41f4cf3f8ee None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.280431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.292229 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.332982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3eaef0-6c00-4ba8-bb06-8356a4ca3fc4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.333893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e28d6-7ddb-4929-9fc7-c41f4cf3f8ee None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.366964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688735b-156e-4c18-aa4e-b865463ec228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.367598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688735b-156e-4c18-aa4e-b865463ec228 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.367880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688735b-156e-4c18-aa4e-b865463ec228 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.368154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688735b-156e-4c18-aa4e-b865463ec228 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.368761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.369252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688735b-156e-4c18-aa4e-b865463ec228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.369696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.370112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.370516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.370641 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688735b-156e-4c18-aa4e-b865463ec228 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.372238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 523/1041] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:21.377682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688735b-156e-4c18-aa4e-b865463ec228 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.384104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.385524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:21.401973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:21.403782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:21.404670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:21.404670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:21.404670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:21.404670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 519/1042] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:21.407299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.490423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688735b-156e-4c18-aa4e-b865463ec228 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.515720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.521166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefe45a-5c86-4258-be7c-f30a91cc7835 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.523450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.525098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc7adf-0ff7-45e1-8dfe-140c4f61fa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.525696 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc7adf-0ff7-45e1-8dfe-140c4f61fa7d None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.525970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc7adf-0ff7-45e1-8dfe-140c4f61fa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.526239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc7adf-0ff7-45e1-8dfe-140c4f61fa7d None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.526875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc7adf-0ff7-45e1-8dfe-140c4f61fa7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.528306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc7adf-0ff7-45e1-8dfe-140c4f61fa7d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.548575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 524/1043] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:47:21 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:47:21.549371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc7adf-0ff7-45e1-8dfe-140c4f61fa7d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.556483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.557103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.557378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.557645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.558327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:21.558624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.560138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.578881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 520/1044] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:21.580226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.668358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc7adf-0ff7-45e1-8dfe-140c4f61fa7d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.694953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19b1cd-31a0-4774-b326-3f4c7e7370dd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.707574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.708220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.708498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.708769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] PATH_INFO: `/v3/services/image` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.709356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.710733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:21.721648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:21.722745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:21.723627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 525/1045] 10.222.0.52 () {56 vars in 1105 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:21.729311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.737033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.737668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.738612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.739042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.747979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.749359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:21.761265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:21.762233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:21.763210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 521/1046] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:21.769847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.784518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.796938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.846081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce001c-ae75-413b-91dc-7081e1a28820 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.881132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb452bf-02cb-4eae-b358-868ef4d199db None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.883611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.884220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.884501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.884775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.885399 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] RBAC: Adding query filter params (name=image) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:21.885702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.887103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.892182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 526/1047] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:21.897713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.922983 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.923653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.923891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.924143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.924825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:21.925126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.927126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.939893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 522/1048] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:21.945636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.011205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e057cc2-8b86-4ada-ab83-8a889acc254c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.022300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:22.037052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:22.047425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.048036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.048327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.048580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.049215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:22.049516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.051464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.055457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9c56b-823f-4d22-83a5-cd6e12bae0ff None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.056997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 527/1049] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:22.062511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.100249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a7c2f-7c05-47a2-add8-4341269ad507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.101007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a7c2f-7c05-47a2-add8-4341269ad507 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.101290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a7c2f-7c05-47a2-add8-4341269ad507 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.101557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a7c2f-7c05-47a2-add8-4341269ad507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.102338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a7c2f-7c05-47a2-add8-4341269ad507 None admin] RBAC: Adding query filter params (role.id=2b49ddcd3d3346a4879bbf33c27d9419, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=152edb19fd7b434485b0e1ebc5078335) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:22.102646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a7c2f-7c05-47a2-add8-4341269ad507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.104225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a7c2f-7c05-47a2-add8-4341269ad507 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.110408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 523/1050] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/role_assignments?user.id=152edb19fd7b434485b0e1ebc5078335&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2b49ddcd3d3346a4879bbf33c27d9419 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:22.185872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c655-acdd-4baa-85d6-e8ef464f8ca7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.222086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb392835-4f01-4264-ac84-a2c91f52a7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.223282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb392835-4f01-4264-ac84-a2c91f52a7f7 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.223564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb392835-4f01-4264-ac84-a2c91f52a7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.223837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb392835-4f01-4264-ac84-a2c91f52a7f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.224512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb392835-4f01-4264-ac84-a2c91f52a7f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.225901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb392835-4f01-4264-ac84-a2c91f52a7f7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.248499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 528/1051] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:47:23.771450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cbb3b-74d4-4698-8cf7-239842640014 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.771961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cbb3b-74d4-4698-8cf7-239842640014 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.771961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cbb3b-74d4-4698-8cf7-239842640014 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.773294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 524/1052] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:23.778396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7a088-a471-4e27-960c-adfced3cf62f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.778769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7a088-a471-4e27-960c-adfced3cf62f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.778983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7a088-a471-4e27-960c-adfced3cf62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.796724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-0ff7a088-a471-4e27-960c-adfced3cf62f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:23.797900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0ff7a088-a471-4e27-960c-adfced3cf62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:23.823954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:23.876279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 529/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:24.148868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cbb3b-74d4-4698-8cf7-239842640014 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.247754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a4c90-d6c8-4afc-97b6-9c42484515d5 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.248037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a4c90-d6c8-4afc-97b6-9c42484515d5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.248312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a4c90-d6c8-4afc-97b6-9c42484515d5 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.249698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 530/1054] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:24.254287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cbb3b-74d4-4698-8cf7-239842640014 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.255117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3671fb-1ff1-4808-bd0d-e033614d70cf None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.255407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3671fb-1ff1-4808-bd0d-e033614d70cf None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.255677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3671fb-1ff1-4808-bd0d-e033614d70cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.274154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-cf3671fb-1ff1-4808-bd0d-e033614d70cf None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:24.275367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-cf3671fb-1ff1-4808-bd0d-e033614d70cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:24.290030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8d1c3-35cd-402e-98b5-3f7217318b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.290633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d1c3-35cd-402e-98b5-3f7217318b7f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.290914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d1c3-35cd-402e-98b5-3f7217318b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.291198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8d1c3-35cd-402e-98b5-3f7217318b7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.301178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8d1c3-35cd-402e-98b5-3f7217318b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-31b8d1c3-35cd-402e-98b5-3f7217318b7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:24.305839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:24.306762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:24.307636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 525/1055] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:24.313259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8d1c3-35cd-402e-98b5-3f7217318b7f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.325153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.357964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 531/1056] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:24.426189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8d1c3-35cd-402e-98b5-3f7217318b7f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.449439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab5d21-60c9-4f12-9b3e-0379cbd459f9 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.449724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab5d21-60c9-4f12-9b3e-0379cbd459f9 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.450089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab5d21-60c9-4f12-9b3e-0379cbd459f9 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.451526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 532/1057] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:24.456384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3b859-7e54-4587-a8c9-b3bf5dcd50fb None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.457241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3b859-7e54-4587-a8c9-b3bf5dcd50fb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.457517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3b859-7e54-4587-a8c9-b3bf5dcd50fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.462206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.462821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.463114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.463383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.464066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:24.464364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.465710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.475330 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 526/1058] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:24.475825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-35d3b859-7e54-4587-a8c9-b3bf5dcd50fb None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:24.477068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-35d3b859-7e54-4587-a8c9-b3bf5dcd50fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:24.481380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.548279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 533/1059] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:24.556224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.588444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d633c52-fdb4-4927-b74f-268ac89dec61 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.622224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1d554-ab32-4dd5-b725-6247a976e252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.622840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1d554-ab32-4dd5-b725-6247a976e252 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.623141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1d554-ab32-4dd5-b725-6247a976e252 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.623408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1d554-ab32-4dd5-b725-6247a976e252 None admin] PATH_INFO: `/v3/users/fe936988ce014e28a17d6e468f506667` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.632664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3b859-7e54-4587-a8c9-b3bf5dcd50fb None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.636691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1d554-ab32-4dd5-b725-6247a976e252 None admin] RBAC: Authorizing `identity:get_user(user_id=fe936988ce014e28a17d6e468f506667)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.638104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1d554-ab32-4dd5-b725-6247a976e252 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.644945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 527/1060] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/users/fe936988ce014e28a17d6e468f506667 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:24.645856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1d554-ab32-4dd5-b725-6247a976e252 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.655116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.740348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3b859-7e54-4587-a8c9-b3bf5dcd50fb None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.762087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1d554-ab32-4dd5-b725-6247a976e252 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.776930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f9117-b3bb-4c37-a183-e00df6c32f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.777575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f9117-b3bb-4c37-a183-e00df6c32f03 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.777855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f9117-b3bb-4c37-a183-e00df6c32f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.778126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f9117-b3bb-4c37-a183-e00df6c32f03 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.789663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f9117-b3bb-4c37-a183-e00df6c32f03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:24.790240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-1f8f9117-b3bb-4c37-a183-e00df6c32f03 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:24.794622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:24.795760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:24.796617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 534/1061] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:24.802186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f9117-b3bb-4c37-a183-e00df6c32f03 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.803389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cf7a8-f594-4949-8f46-11e6c18f0478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.803996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cf7a8-f594-4949-8f46-11e6c18f0478 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.804280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cf7a8-f594-4949-8f46-11e6c18f0478 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.804548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cf7a8-f594-4949-8f46-11e6c18f0478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.880668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.886346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cf7a8-f594-4949-8f46-11e6c18f0478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.887849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cf7a8-f594-4949-8f46-11e6c18f0478 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.903602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f9117-b3bb-4c37-a183-e00df6c32f03 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.938076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.938610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.938910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.939188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.939973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:24.940286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.941681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.949556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 535/1062] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:24.950328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.989898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 528/1063] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 11:47:24.990740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cf7a8-f594-4949-8f46-11e6c18f0478 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.018783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.054796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605ef05-ca80-4464-a99a-f066b2bca248 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.089728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e8f24-bf4a-4c9b-b314-c81622777ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.090510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e8f24-bf4a-4c9b-b314-c81622777ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.090973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e8f24-bf4a-4c9b-b314-c81622777ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.091256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e8f24-bf4a-4c9b-b314-c81622777ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.093762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cf7a8-f594-4949-8f46-11e6c18f0478 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.101320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.129385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.130004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.130279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.130545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.140436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.141881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:25.156379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:25.157319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:25.158223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:25.158223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:25.158223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:25.158223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 529/1064] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:25.158223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.160962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e8f24-bf4a-4c9b-b314-c81622777ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.162586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e8f24-bf4a-4c9b-b314-c81622777ec7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.238529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.257858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfeeb0-1771-4b8b-b194-37188f0ad939 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.261986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 536/1065] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:25.262841 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e8f24-bf4a-4c9b-b314-c81622777ec7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.294952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff17c7-e825-49b6-b2a6-8178b2db0462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.295577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff17c7-e825-49b6-b2a6-8178b2db0462 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.295854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff17c7-e825-49b6-b2a6-8178b2db0462 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.296123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff17c7-e825-49b6-b2a6-8178b2db0462 None admin] PATH_INFO: `/v3/projects/5d233b8995704975a4a6d81e6f55cbd4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.297230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff17c7-e825-49b6-b2a6-8178b2db0462 None admin] RBAC: Authorizing `identity:get_project(project_id=5d233b8995704975a4a6d81e6f55cbd4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.298628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff17c7-e825-49b6-b2a6-8178b2db0462 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.300301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 530/1066] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/projects/5d233b8995704975a4a6d81e6f55cbd4 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:25.301054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff17c7-e825-49b6-b2a6-8178b2db0462 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.301553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.365156 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e8f24-bf4a-4c9b-b314-c81622777ec7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.399150 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.399451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff17c7-e825-49b6-b2a6-8178b2db0462 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.399808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.400028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.400296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.400926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:25.401225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.402637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.417254 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 537/1067] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:25.418070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.444651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.445280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.445560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.445833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.456342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.458295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.466160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:25.474189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:25.475329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:25.476285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:25.476285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:25.476285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:25.476285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 531/1068] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:25.476509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.517677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.522304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e9b8e-ba58-43d1-87af-4ef6b23a4a83 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.555524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.556121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.556393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.556667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.557377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=5d233b8995704975a4a6d81e6f55cbd4, user.id=fe936988ce014e28a17d6e468f506667) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:25.557677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.559115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.564528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 538/1069] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/role_assignments?user.id=fe936988ce014e28a17d6e468f506667&scope.project.id=5d233b8995704975a4a6d81e6f55cbd4&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:25.565284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.582995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b92eb-3d80-4823-8023-0880beb895b6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.621522 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.622175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.622452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.622720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.633018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.634478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:25.647408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:25.648438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:25.649457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 532/1070] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:25.654557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.683724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa6001-06e7-44b8-8d5b-967674d8e727 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.718974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.721947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.722679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.722985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.723259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.723903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:25.724211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.725589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.738097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 539/1071] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:25.766588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04df66-ac54-42e4-b9d8-5ffece7f581e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.804372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.804987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.805211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.805431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.806168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:25.806467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.807908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.833642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 533/1072] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:25.835162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.861439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.944768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad091b2-8284-4637-821f-b05919b9fde4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.983797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.984432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.984710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.984978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.996529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.997981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:26.014404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:26.015515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:26.016389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:26.016389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:26.016389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:26.016389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 540/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:26.022577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:26.126795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9251f-201a-4202-98d0-02c31a525e42 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:26.144338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:26.161413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:26.162010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:26.162285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:26.162552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:26.163208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:26.163533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:26.164863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.176259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 534/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:26.182042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:26.254670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:26.278448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3956561-1559-43d9-81ff-cc8a265855e0 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:26.310902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:26.311546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:26.312075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:26.312342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:26.319320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:26.320797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:26.331444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:26.332321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:26.333221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 541/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:26.338281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:26.436593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f2e4b-a035-4027-853e-42f6bd68d9cf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:26.469924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8db000-c6aa-4fa7-9cb7-a0cef5ff0b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:26.470532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db000-c6aa-4fa7-9cb7-a0cef5ff0b2a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:26.470810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db000-c6aa-4fa7-9cb7-a0cef5ff0b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:26.471107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db000-c6aa-4fa7-9cb7-a0cef5ff0b2a None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:26.471804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8db000-c6aa-4fa7-9cb7-a0cef5ff0b2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:26.472656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8db000-c6aa-4fa7-9cb7-a0cef5ff0b2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:26.474060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8db000-c6aa-4fa7-9cb7-a0cef5ff0b2a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.486084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 535/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:26.492517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:26.587704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8336be-1f66-4c87-855e-ca96fecdb175 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:26.625251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc455-5362-44d3-9861-7646b181d33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:26.625901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc455-5362-44d3-9861-7646b181d33b None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:26.626182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc455-5362-44d3-9861-7646b181d33b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:26.626453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc455-5362-44d3-9861-7646b181d33b None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/152edb19fd7b434485b0e1ebc5078335/roles/2b49ddcd3d3346a4879bbf33c27d9419` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:26.628594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbc455-5362-44d3-9861-7646b181d33b None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=152edb19fd7b434485b0e1ebc5078335, role_id=2b49ddcd3d3346a4879bbf33c27d9419)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:26.630031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbc455-5362-44d3-9861-7646b181d33b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.632242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:26.643465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 542/1077] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:26 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/152edb19fd7b434485b0e1ebc5078335/roles/2b49ddcd3d3346a4879bbf33c27d9419 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:47:27.746949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e983-1d55-4ea3-894a-a1b90ad6a8cb None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:27.747469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e983-1d55-4ea3-894a-a1b90ad6a8cb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:27.747469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e983-1d55-4ea3-894a-a1b90ad6a8cb None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:27.748739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 536/1078] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:27.754589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a8d3-1039-44eb-8b1e-461250c3c64d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:27.755054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a8d3-1039-44eb-8b1e-461250c3c64d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:27.755177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a8d3-1039-44eb-8b1e-461250c3c64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:27.772876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5e40a8d3-1039-44eb-8b1e-461250c3c64d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:27.774043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5e40a8d3-1039-44eb-8b1e-461250c3c64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:27.838301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 543/1079] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:28.110761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e983-1d55-4ea3-894a-a1b90ad6a8cb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:28.150453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:28.209743 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e983-1d55-4ea3-894a-a1b90ad6a8cb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:28.242708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017ee0-cefc-41e6-95c2-fe56f1a50454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:28.243355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017ee0-cefc-41e6-95c2-fe56f1a50454 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.243634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017ee0-cefc-41e6-95c2-fe56f1a50454 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.243902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017ee0-cefc-41e6-95c2-fe56f1a50454 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.253649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017ee0-cefc-41e6-95c2-fe56f1a50454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-dd017ee0-cefc-41e6-95c2-fe56f1a50454 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:28.258328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:28.259511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:28.260432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 537/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:28.265431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40a8d3-1039-44eb-8b1e-461250c3c64d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:28.362104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40a8d3-1039-44eb-8b1e-461250c3c64d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:28.384667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:28.396442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:28.397049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.397327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.397599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.398266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:28.398572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:28.400001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:28.408587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 544/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:28.415112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017ee0-cefc-41e6-95c2-fe56f1a50454 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:28.508664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:28.512727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017ee0-cefc-41e6-95c2-fe56f1a50454 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:28.546936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f57009-0748-41a4-bec6-382bec10e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:28.547558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57009-0748-41a4-bec6-382bec10e590 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.547839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57009-0748-41a4-bec6-382bec10e590 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.548115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57009-0748-41a4-bec6-382bec10e590 None admin] PATH_INFO: `/v3/users/fe936988ce014e28a17d6e468f506667` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.549233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f57009-0748-41a4-bec6-382bec10e590 None admin] RBAC: Authorizing `identity:get_user(user_id=fe936988ce014e28a17d6e468f506667)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:28.550655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f57009-0748-41a4-bec6-382bec10e590 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:28.552327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 538/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:28 2022] GET /identity/v3/users/fe936988ce014e28a17d6e468f506667 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:28.558040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:28.656616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2056f-df17-449e-a3b0-9fb080b39481 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:28.693449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddce85-3ca9-4688-9d0b-721d5238aebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:28.694085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddce85-3ca9-4688-9d0b-721d5238aebc None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.694365 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddce85-3ca9-4688-9d0b-721d5238aebc None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.694642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddce85-3ca9-4688-9d0b-721d5238aebc None admin] PATH_INFO: `/v3/projects/5d233b8995704975a4a6d81e6f55cbd4` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.695804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddce85-3ca9-4688-9d0b-721d5238aebc None admin] RBAC: Authorizing `identity:get_project(project_id=5d233b8995704975a4a6d81e6f55cbd4)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:28.697230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddce85-3ca9-4688-9d0b-721d5238aebc None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:28.698862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 545/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:47:28 2022] GET /identity/v3/projects/5d233b8995704975a4a6d81e6f55cbd4 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:28.704723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f57009-0748-41a4-bec6-382bec10e590 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:28.776819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42450-a7d5-4ae9-a8b3-4043eec45dfc None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.777379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42450-a7d5-4ae9-a8b3-4043eec45dfc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.777379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42450-a7d5-4ae9-a8b3-4043eec45dfc None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.778700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 546/1084] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:28.783836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678194c-9338-4374-95d9-c05e45accec5 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.784119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678194c-9338-4374-95d9-c05e45accec5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.784388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678194c-9338-4374-95d9-c05e45accec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.802152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f57009-0748-41a4-bec6-382bec10e590 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:28.802624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f678194c-9338-4374-95d9-c05e45accec5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:28.803565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f678194c-9338-4374-95d9-c05e45accec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:28.836778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d16280-f99c-4a25-ba90-5395c6baf6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:28.837434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16280-f99c-4a25-ba90-5395c6baf6b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.837714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16280-f99c-4a25-ba90-5395c6baf6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.837990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16280-f99c-4a25-ba90-5395c6baf6b3 None admin] PATH_INFO: `/v3/projects/5d233b8995704975a4a6d81e6f55cbd4/users/fe936988ce014e28a17d6e468f506667/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.839926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d16280-f99c-4a25-ba90-5395c6baf6b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d233b8995704975a4a6d81e6f55cbd4, user_id=fe936988ce014e28a17d6e468f506667, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:28.841344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d16280-f99c-4a25-ba90-5395c6baf6b3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:28.852818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:28.854417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 539/1085] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:28 2022] PUT /identity/v3/projects/5d233b8995704975a4a6d81e6f55cbd4/users/fe936988ce014e28a17d6e468f506667/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:47:28.871450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 547/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:29.137792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d16280-f99c-4a25-ba90-5395c6baf6b3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.164357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:29.236664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d16280-f99c-4a25-ba90-5395c6baf6b3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.269878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ac8a-e8b1-4edb-a8cc-17a3925d48e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.270491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ac8a-e8b1-4edb-a8cc-17a3925d48e8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.270763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ac8a-e8b1-4edb-a8cc-17a3925d48e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.271056 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ac8a-e8b1-4edb-a8cc-17a3925d48e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.280782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6ac8a-e8b1-4edb-a8cc-17a3925d48e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-82e6ac8a-e8b1-4edb-a8cc-17a3925d48e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:29.285573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:29.286505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:29.287410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:29.291034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 540/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:29.292548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678194c-9338-4374-95d9-c05e45accec5 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.391823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678194c-9338-4374-95d9-c05e45accec5 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.427702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.428302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.428576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.428839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.429488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:29.429782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:29.431274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:29.439554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 548/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:29.445421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ac8a-e8b1-4edb-a8cc-17a3925d48e8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.526883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:29.542208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ac8a-e8b1-4edb-a8cc-17a3925d48e8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.575673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.576350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.576629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.576899 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.586642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:29.588103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:29.602653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:29.603781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:29.604900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:29.604900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:29.604900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:29.604900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 541/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:29.609571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.654264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:29.706278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f3589-f439-47e4-bb9b-9c71298a4b8c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.741225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.741827 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.742102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.742368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.743021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:29.743323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:29.744663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:29.755929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 549/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:29.761671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.858352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2939d3-89f1-458a-87fd-047aa86eb94a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.891778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.892386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.892659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.892924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.900113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:29.901520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:29.903202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:29.913747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:29.914939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:29.916121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:29.919034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 542/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:29.920551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.020718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d418-fcfe-4a82-9bd3-e4d6f21cd39d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.028755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:30.056137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.056734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.057014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.057281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.057994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:30.058294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.059734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:30.077213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 550/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:30.083668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.179621 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3dfec-c5cc-4651-b64f-a0760be89da6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.212199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacea1c0-ebcf-4671-9159-e18ed16bcf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.212789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea1c0-ebcf-4671-9159-e18ed16bcf8b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.213070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea1c0-ebcf-4671-9159-e18ed16bcf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.213338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea1c0-ebcf-4671-9159-e18ed16bcf8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.273554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:30.277050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacea1c0-ebcf-4671-9159-e18ed16bcf8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.278629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacea1c0-ebcf-4671-9159-e18ed16bcf8b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:30.370383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 543/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:30.376091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.475448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa77c0-890b-4e5c-9af2-2094be4efbcd None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.508764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.509387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.509662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.509929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.519621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.521041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:30.535686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:30.536785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:30.538153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:30.538153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:30.538153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:30.538153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 551/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:30.542384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacea1c0-ebcf-4671-9159-e18ed16bcf8b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.636912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:30.643464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacea1c0-ebcf-4671-9159-e18ed16bcf8b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.676664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.677255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.677570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.677841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.678469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:30.678764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.680128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:30.691585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 544/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:30.697081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.715521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:30.794480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb2742-4cce-4708-9b0a-b5ad3a70d8f7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.827017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7bebd-54c3-4e4a-be86-819bcadab846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.827633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7bebd-54c3-4e4a-be86-819bcadab846 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.827906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7bebd-54c3-4e4a-be86-819bcadab846 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.828167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7bebd-54c3-4e4a-be86-819bcadab846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.835079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7bebd-54c3-4e4a-be86-819bcadab846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.836509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7bebd-54c3-4e4a-be86-819bcadab846 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-55b7bebd-54c3-4e4a-be86-819bcadab846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:30.848028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:30.849000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:30.850308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 552/1096] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:30.854882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.956932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54c924-5e87-4838-b161-89ec625cc79c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.983110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9dbbc-54fe-4edb-9ed5-e925abb0188e None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.983624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9dbbc-54fe-4edb-9ed5-e925abb0188e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.983624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9dbbc-54fe-4edb-9ed5-e925abb0188e None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.984965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 553/1097] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:30.989145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9258ea-5847-4747-9223-80011193439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.989744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9258ea-5847-4747-9223-80011193439e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.990017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9258ea-5847-4747-9223-80011193439e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.990286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9258ea-5847-4747-9223-80011193439e None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.990382 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33247b2-4204-4c99-bacf-06700c4f2f13 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.990635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33247b2-4204-4c99-bacf-06700c4f2f13 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.990932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33247b2-4204-4c99-bacf-06700c4f2f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.991042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9258ea-5847-4747-9223-80011193439e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:30.991288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9258ea-5847-4747-9223-80011193439e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.992685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9258ea-5847-4747-9223-80011193439e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.004835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 545/1098] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.009175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c33247b2-4204-4c99-bacf-06700c4f2f13 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:31.010360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c33247b2-4204-4c99-bacf-06700c4f2f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:31.010612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9258ea-5847-4747-9223-80011193439e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.029175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.078060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 554/1099] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:31.108314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9258ea-5847-4747-9223-80011193439e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.140548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.141138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.141413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.141678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.142387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] RBAC: Adding query filter params (role.id=2b49ddcd3d3346a4879bbf33c27d9419, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=152edb19fd7b434485b0e1ebc5078335) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:31.142685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.144163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.149649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 546/1100] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/role_assignments?user.id=152edb19fd7b434485b0e1ebc5078335&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2b49ddcd3d3346a4879bbf33c27d9419 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.351199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33247b2-4204-4c99-bacf-06700c4f2f13 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.359343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.459146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33247b2-4204-4c99-bacf-06700c4f2f13 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.492995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d40529-e39f-4c1c-a00c-3f3cc6c14794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.493619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d40529-e39f-4c1c-a00c-3f3cc6c14794 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.493892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d40529-e39f-4c1c-a00c-3f3cc6c14794 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.494160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d40529-e39f-4c1c-a00c-3f3cc6c14794 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.504555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d40529-e39f-4c1c-a00c-3f3cc6c14794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-15d40529-e39f-4c1c-a00c-3f3cc6c14794 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:31.509374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:31.510370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:31.511278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 555/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:31.516746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.600654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.619847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089eae5a-44ab-4f8a-9845-79e0b92fc19f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.653582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.654189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.654465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.654733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.655408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:31.655704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.657039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.665591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 547/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.671627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d40529-e39f-4c1c-a00c-3f3cc6c14794 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.738746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.774363 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d40529-e39f-4c1c-a00c-3f3cc6c14794 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.807936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d37601-1caf-46e0-98d2-356285c97341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.808557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d37601-1caf-46e0-98d2-356285c97341 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.808831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d37601-1caf-46e0-98d2-356285c97341 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.809095 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d37601-1caf-46e0-98d2-356285c97341 None admin] PATH_INFO: `/v3/users/fe936988ce014e28a17d6e468f506667` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.810195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d37601-1caf-46e0-98d2-356285c97341 None admin] RBAC: Authorizing `identity:get_user(user_id=fe936988ce014e28a17d6e468f506667)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.811892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d37601-1caf-46e0-98d2-356285c97341 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.813447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 556/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/users/fe936988ce014e28a17d6e468f506667 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.819890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.921197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78540062-4c52-4bec-bd49-1992dfecdc08 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.954723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7867089-a283-480a-ac46-150153d2b354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.955347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867089-a283-480a-ac46-150153d2b354 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.955624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867089-a283-480a-ac46-150153d2b354 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.955892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867089-a283-480a-ac46-150153d2b354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.970600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.024826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7867089-a283-480a-ac46-150153d2b354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.026254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7867089-a283-480a-ac46-150153d2b354 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.120614 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 548/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:32.126615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d37601-1caf-46e0-98d2-356285c97341 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.221449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d37601-1caf-46e0-98d2-356285c97341 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.252721 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8066cc-235e-45e0-bbf6-e7f3b7a7d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.253892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8066cc-235e-45e0-bbf6-e7f3b7a7d80b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.254168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8066cc-235e-45e0-bbf6-e7f3b7a7d80b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.254432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8066cc-235e-45e0-bbf6-e7f3b7a7d80b None admin] PATH_INFO: `/v3/projects/5d233b8995704975a4a6d81e6f55cbd4` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.255539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8066cc-235e-45e0-bbf6-e7f3b7a7d80b None admin] RBAC: Authorizing `identity:get_project(project_id=5d233b8995704975a4a6d81e6f55cbd4)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.256936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8066cc-235e-45e0-bbf6-e7f3b7a7d80b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.258498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 557/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/projects/5d233b8995704975a4a6d81e6f55cbd4 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:32.264340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7867089-a283-480a-ac46-150153d2b354 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.307135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.358112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7867089-a283-480a-ac46-150153d2b354 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.390384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975bf70-3121-4c1d-99a1-29867410f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.390993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975bf70-3121-4c1d-99a1-29867410f2ce None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.391268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975bf70-3121-4c1d-99a1-29867410f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.391531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975bf70-3121-4c1d-99a1-29867410f2ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.392236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975bf70-3121-4c1d-99a1-29867410f2ce None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=5d233b8995704975a4a6d81e6f55cbd4, user.id=fe936988ce014e28a17d6e468f506667) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:32.392535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975bf70-3121-4c1d-99a1-29867410f2ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.393942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975bf70-3121-4c1d-99a1-29867410f2ce None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.399394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 549/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/role_assignments?user.id=fe936988ce014e28a17d6e468f506667&scope.project.id=5d233b8995704975a4a6d81e6f55cbd4&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:34.538222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d398116-c3aa-4d97-ba36-b8a2460d5770 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:34.538756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d398116-c3aa-4d97-ba36-b8a2460d5770 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:34.538756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d398116-c3aa-4d97-ba36-b8a2460d5770 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:34.540106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 558/1107] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:34.545173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7501c-6d60-4333-8dd1-00508f8c9e59 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:34.545569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7501c-6d60-4333-8dd1-00508f8c9e59 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:34.545708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7501c-6d60-4333-8dd1-00508f8c9e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:34.564100 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-74c7501c-6d60-4333-8dd1-00508f8c9e59 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:34.565261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-74c7501c-6d60-4333-8dd1-00508f8c9e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:34.629126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 550/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:34.894797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d398116-c3aa-4d97-ba36-b8a2460d5770 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:34.897677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:34.993517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d398116-c3aa-4d97-ba36-b8a2460d5770 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:35.026036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e97171-ee04-4768-83c2-8abdddb6972a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:35.026648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97171-ee04-4768-83c2-8abdddb6972a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:35.026945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97171-ee04-4768-83c2-8abdddb6972a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:35.027214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e97171-ee04-4768-83c2-8abdddb6972a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:35.028303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e97171-ee04-4768-83c2-8abdddb6972a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:35.029711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e97171-ee04-4768-83c2-8abdddb6972a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:35.031258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 559/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:35.037335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7501c-6d60-4333-8dd1-00508f8c9e59 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:35.075036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:35.132811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7501c-6d60-4333-8dd1-00508f8c9e59 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:35.165266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd684b-7efb-47ab-bc2c-632807a922e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:35.165797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd684b-7efb-47ab-bc2c-632807a922e3 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:35.166072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd684b-7efb-47ab-bc2c-632807a922e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:35.166336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd684b-7efb-47ab-bc2c-632807a922e3 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:35.167013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd684b-7efb-47ab-bc2c-632807a922e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:35.168403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd684b-7efb-47ab-bc2c-632807a922e3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:35.188977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 551/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:47:35 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:47:37.357905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fa792-d6a4-4a40-af84-4131b625f8e0 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.358429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fa792-d6a4-4a40-af84-4131b625f8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.358539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fa792-d6a4-4a40-af84-4131b625f8e0 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.359840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 560/1111] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:37.365651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd6800-eba2-4ae3-9944-d404e02132c6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.366028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd6800-eba2-4ae3-9944-d404e02132c6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.366209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd6800-eba2-4ae3-9944-d404e02132c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.385203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-decd6800-eba2-4ae3-9944-d404e02132c6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:37.386497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-decd6800-eba2-4ae3-9944-d404e02132c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:37.451814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 552/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:37.719093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fa792-d6a4-4a40-af84-4131b625f8e0 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.786002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:37.816980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fa792-d6a4-4a40-af84-4131b625f8e0 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.848980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c3092-a1f1-4329-b2ca-615373bcf28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.849584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c3092-a1f1-4329-b2ca-615373bcf28e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.849856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c3092-a1f1-4329-b2ca-615373bcf28e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.850121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c3092-a1f1-4329-b2ca-615373bcf28e None admin] PATH_INFO: `/v3/roles/2e40648646094078acac22f8eb94dd98` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.851497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c3092-a1f1-4329-b2ca-615373bcf28e None admin] RBAC: Authorizing `identity:get_role(role_id=2e40648646094078acac22f8eb94dd98)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:37.852832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c3092-a1f1-4329-b2ca-615373bcf28e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:37.854104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 561/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/roles/2e40648646094078acac22f8eb94dd98 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:37.859770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd6800-eba2-4ae3-9944-d404e02132c6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.883654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:37.961897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd6800-eba2-4ae3-9944-d404e02132c6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.994658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da23cf-074b-4c29-a729-9b60f17fe8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.995243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da23cf-074b-4c29-a729-9b60f17fe8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.995521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da23cf-074b-4c29-a729-9b60f17fe8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.995789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da23cf-074b-4c29-a729-9b60f17fe8b1 None admin] PATH_INFO: `/v3/users/6b34e532a7154f229a5d9670c5743675` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.008124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da23cf-074b-4c29-a729-9b60f17fe8b1 None admin] RBAC: Authorizing `identity:get_user(user_id=6b34e532a7154f229a5d9670c5743675)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.009537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da23cf-074b-4c29-a729-9b60f17fe8b1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.011160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 553/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/users/6b34e532a7154f229a5d9670c5743675 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:38.016895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c3092-a1f1-4329-b2ca-615373bcf28e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.115044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c3092-a1f1-4329-b2ca-615373bcf28e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.146754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab08797-8841-4c56-8044-88cd69f80704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.147370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab08797-8841-4c56-8044-88cd69f80704 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.147647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab08797-8841-4c56-8044-88cd69f80704 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.147912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab08797-8841-4c56-8044-88cd69f80704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.155930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:38.211540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab08797-8841-4c56-8044-88cd69f80704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.212962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab08797-8841-4c56-8044-88cd69f80704 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.305950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 562/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:38.312115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da23cf-074b-4c29-a729-9b60f17fe8b1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.391085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:38.408876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da23cf-074b-4c29-a729-9b60f17fe8b1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.442501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de491696-22af-4838-8f86-04d7a8b235cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.443150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de491696-22af-4838-8f86-04d7a8b235cd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.443429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de491696-22af-4838-8f86-04d7a8b235cd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.443700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de491696-22af-4838-8f86-04d7a8b235cd None admin] PATH_INFO: `/v3/projects/5d233b8995704975a4a6d81e6f55cbd4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.444797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de491696-22af-4838-8f86-04d7a8b235cd None admin] RBAC: Authorizing `identity:get_project(project_id=5d233b8995704975a4a6d81e6f55cbd4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.446220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de491696-22af-4838-8f86-04d7a8b235cd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.448003 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 554/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/projects/5d233b8995704975a4a6d81e6f55cbd4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:38.454053 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab08797-8841-4c56-8044-88cd69f80704 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.493094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:38.550603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab08797-8841-4c56-8044-88cd69f80704 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.584074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78213b-2a3c-4af0-a1cd-bd4724709563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.584663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78213b-2a3c-4af0-a1cd-bd4724709563 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.584942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78213b-2a3c-4af0-a1cd-bd4724709563 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.585209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78213b-2a3c-4af0-a1cd-bd4724709563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.585922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78213b-2a3c-4af0-a1cd-bd4724709563 None admin] RBAC: Adding query filter params (role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=5d233b8995704975a4a6d81e6f55cbd4, user.id=6b34e532a7154f229a5d9670c5743675) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:38.586224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78213b-2a3c-4af0-a1cd-bd4724709563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.587694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78213b-2a3c-4af0-a1cd-bd4724709563 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.592857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 563/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/role_assignments?user.id=6b34e532a7154f229a5d9670c5743675&scope.project.id=5d233b8995704975a4a6d81e6f55cbd4&role.id=2e40648646094078acac22f8eb94dd98 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:40.673014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1742f-a299-48c3-baa0-70605c0f1e67 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:40.673536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1742f-a299-48c3-baa0-70605c0f1e67 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:40.673536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1742f-a299-48c3-baa0-70605c0f1e67 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:40.675483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 555/1118] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:40.680547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67997-0e6e-4c38-94cb-429d0f1388f6 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:40.680971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67997-0e6e-4c38-94cb-429d0f1388f6 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:40.681088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67997-0e6e-4c38-94cb-429d0f1388f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:40.699398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e9d67997-0e6e-4c38-94cb-429d0f1388f6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:40.700570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e9d67997-0e6e-4c38-94cb-429d0f1388f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:40.764278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 564/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:41.030229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1742f-a299-48c3-baa0-70605c0f1e67 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:41.126940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1742f-a299-48c3-baa0-70605c0f1e67 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:41.158809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904632a1-396e-41df-94a3-459a0ce07321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:41.159984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904632a1-396e-41df-94a3-459a0ce07321 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:41.160258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904632a1-396e-41df-94a3-459a0ce07321 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:41.160526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904632a1-396e-41df-94a3-459a0ce07321 None admin] PATH_INFO: `/v3/roles/2e40648646094078acac22f8eb94dd98` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:41.161893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904632a1-396e-41df-94a3-459a0ce07321 None admin] RBAC: Authorizing `identity:get_role(role_id=2e40648646094078acac22f8eb94dd98)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:41.163259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904632a1-396e-41df-94a3-459a0ce07321 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:41.164537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 556/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:41 2022] GET /identity/v3/roles/2e40648646094078acac22f8eb94dd98 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:41.170169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d67997-0e6e-4c38-94cb-429d0f1388f6 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:41.199370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:41.265408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d67997-0e6e-4c38-94cb-429d0f1388f6 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:41.298426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbeb77-0d06-4cac-b42d-fc53f529f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:41.299024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbeb77-0d06-4cac-b42d-fc53f529f3ad None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:41.299290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbeb77-0d06-4cac-b42d-fc53f529f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:41.299582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbeb77-0d06-4cac-b42d-fc53f529f3ad None admin] PATH_INFO: `/v3/users/6b34e532a7154f229a5d9670c5743675` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:41.300676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbeb77-0d06-4cac-b42d-fc53f529f3ad None admin] RBAC: Authorizing `identity:get_user(user_id=6b34e532a7154f229a5d9670c5743675)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:41.302086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbeb77-0d06-4cac-b42d-fc53f529f3ad None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:41.303645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 565/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:41 2022] GET /identity/v3/users/6b34e532a7154f229a5d9670c5743675 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:41.309178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904632a1-396e-41df-94a3-459a0ce07321 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:41.327642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:41.406050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904632a1-396e-41df-94a3-459a0ce07321 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:41.439097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852bca3-8184-4439-998d-5d9e4cd66545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:41.439709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852bca3-8184-4439-998d-5d9e4cd66545 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:41.439988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852bca3-8184-4439-998d-5d9e4cd66545 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:41.440787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852bca3-8184-4439-998d-5d9e4cd66545 None admin] PATH_INFO: `/v3/projects/5d233b8995704975a4a6d81e6f55cbd4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:41.441891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852bca3-8184-4439-998d-5d9e4cd66545 None admin] RBAC: Authorizing `identity:get_project(project_id=5d233b8995704975a4a6d81e6f55cbd4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:41.443322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852bca3-8184-4439-998d-5d9e4cd66545 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:41.444927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 557/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:47:41 2022] GET /identity/v3/projects/5d233b8995704975a4a6d81e6f55cbd4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:41.450814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbeb77-0d06-4cac-b42d-fc53f529f3ad None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:41.543314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:41.546294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbeb77-0d06-4cac-b42d-fc53f529f3ad None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:41.579113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243dcd8-5632-4ebe-994f-000fecfac7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:41.579753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243dcd8-5632-4ebe-994f-000fecfac7ae None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:41.580032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243dcd8-5632-4ebe-994f-000fecfac7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:41.580306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243dcd8-5632-4ebe-994f-000fecfac7ae None admin] PATH_INFO: `/v3/projects/5d233b8995704975a4a6d81e6f55cbd4/users/6b34e532a7154f229a5d9670c5743675/roles/2e40648646094078acac22f8eb94dd98` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:41.582926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243dcd8-5632-4ebe-994f-000fecfac7ae None admin] RBAC: Authorizing `identity:create_grant(project_id=5d233b8995704975a4a6d81e6f55cbd4, user_id=6b34e532a7154f229a5d9670c5743675, role_id=2e40648646094078acac22f8eb94dd98)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:41.584334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243dcd8-5632-4ebe-994f-000fecfac7ae None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:41.596180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 566/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:41 2022] PUT /identity/v3/projects/5d233b8995704975a4a6d81e6f55cbd4/users/6b34e532a7154f229a5d9670c5743675/roles/2e40648646094078acac22f8eb94dd98 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:47:43.699875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a147aec-47b2-43b5-b08f-ab92d49fa2b4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:43.700396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a147aec-47b2-43b5-b08f-ab92d49fa2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:43.700396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a147aec-47b2-43b5-b08f-ab92d49fa2b4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:43.701664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 558/1124] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:43.706790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c34d2-0381-4d77-b717-a8f052c71720 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:43.707256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c34d2-0381-4d77-b717-a8f052c71720 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:43.707348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c34d2-0381-4d77-b717-a8f052c71720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:43.725358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-634c34d2-0381-4d77-b717-a8f052c71720 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:43.726516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-634c34d2-0381-4d77-b717-a8f052c71720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:43.791735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 567/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:44.058057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a147aec-47b2-43b5-b08f-ab92d49fa2b4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:44.140740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:44.156858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a147aec-47b2-43b5-b08f-ab92d49fa2b4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:44.190246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c673673-a3bf-43e3-878e-3381b6cd787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:44.190860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673673-a3bf-43e3-878e-3381b6cd787b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.191165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673673-a3bf-43e3-878e-3381b6cd787b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.191437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673673-a3bf-43e3-878e-3381b6cd787b None admin] PATH_INFO: `/v3/roles/2e40648646094078acac22f8eb94dd98` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.192811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c673673-a3bf-43e3-878e-3381b6cd787b None admin] RBAC: Authorizing `identity:get_role(role_id=2e40648646094078acac22f8eb94dd98)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:44.194153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c673673-a3bf-43e3-878e-3381b6cd787b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:44.195477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 559/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:44 2022] GET /identity/v3/roles/2e40648646094078acac22f8eb94dd98 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:44.201211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c34d2-0381-4d77-b717-a8f052c71720 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:44.286484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:44.296814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c34d2-0381-4d77-b717-a8f052c71720 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:44.328611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc053e-ed49-4eec-9b9c-96d0da534ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:44.329222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc053e-ed49-4eec-9b9c-96d0da534ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.329495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc053e-ed49-4eec-9b9c-96d0da534ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.329761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc053e-ed49-4eec-9b9c-96d0da534ed1 None admin] PATH_INFO: `/v3/users/6b34e532a7154f229a5d9670c5743675` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.331383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc053e-ed49-4eec-9b9c-96d0da534ed1 None admin] RBAC: Authorizing `identity:get_user(user_id=6b34e532a7154f229a5d9670c5743675)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:44.332786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc053e-ed49-4eec-9b9c-96d0da534ed1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:44.334310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 568/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:44 2022] GET /identity/v3/users/6b34e532a7154f229a5d9670c5743675 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:44.339901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c673673-a3bf-43e3-878e-3381b6cd787b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:44.435272 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c673673-a3bf-43e3-878e-3381b6cd787b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:44.468426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340521b-bdeb-42d0-8c5b-f29c7b6294d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:44.469013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340521b-bdeb-42d0-8c5b-f29c7b6294d4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.469288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340521b-bdeb-42d0-8c5b-f29c7b6294d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.469551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340521b-bdeb-42d0-8c5b-f29c7b6294d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.491634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:44.533644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340521b-bdeb-42d0-8c5b-f29c7b6294d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:44.535138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340521b-bdeb-42d0-8c5b-f29c7b6294d4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:44.627020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 560/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:44.633339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc053e-ed49-4eec-9b9c-96d0da534ed1 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:44.732002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc053e-ed49-4eec-9b9c-96d0da534ed1 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:44.767151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6646409-e495-4263-b46a-7de80b80e7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:44.767836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6646409-e495-4263-b46a-7de80b80e7aa None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.768111 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6646409-e495-4263-b46a-7de80b80e7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.768410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6646409-e495-4263-b46a-7de80b80e7aa None admin] PATH_INFO: `/v3/projects/5d233b8995704975a4a6d81e6f55cbd4` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.769491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6646409-e495-4263-b46a-7de80b80e7aa None admin] RBAC: Authorizing `identity:get_project(project_id=5d233b8995704975a4a6d81e6f55cbd4)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:44.770912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6646409-e495-4263-b46a-7de80b80e7aa None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:44.772481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 569/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:47:44 2022] GET /identity/v3/projects/5d233b8995704975a4a6d81e6f55cbd4 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:44.778394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340521b-bdeb-42d0-8c5b-f29c7b6294d4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:44.832121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:44.874490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340521b-bdeb-42d0-8c5b-f29c7b6294d4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:44.906697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a462ae2-63a7-4576-8893-fdd9a5b1a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:44.907249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a462ae2-63a7-4576-8893-fdd9a5b1a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.907526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a462ae2-63a7-4576-8893-fdd9a5b1a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.907793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a462ae2-63a7-4576-8893-fdd9a5b1a0f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.908498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a462ae2-63a7-4576-8893-fdd9a5b1a0f5 None admin] RBAC: Adding query filter params (role.id=2e40648646094078acac22f8eb94dd98, scope.project.id=5d233b8995704975a4a6d81e6f55cbd4, user.id=6b34e532a7154f229a5d9670c5743675) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:44.908796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a462ae2-63a7-4576-8893-fdd9a5b1a0f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:44.910192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a462ae2-63a7-4576-8893-fdd9a5b1a0f5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:44.915706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 561/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:44 2022] GET /identity/v3/role_assignments?user.id=6b34e532a7154f229a5d9670c5743675&scope.project.id=5d233b8995704975a4a6d81e6f55cbd4&role.id=2e40648646094078acac22f8eb94dd98 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:47.058838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441eb0-e0ed-4b61-8607-4a4f518df472 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.059434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441eb0-e0ed-4b61-8607-4a4f518df472 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.059434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441eb0-e0ed-4b61-8607-4a4f518df472 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.060723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 570/1131] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:47.066251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c7b4e-4e4f-4fe6-b868-c3f9d300bd99 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.066652 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c7b4e-4e4f-4fe6-b868-c3f9d300bd99 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.066798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c7b4e-4e4f-4fe6-b868-c3f9d300bd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.085326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a15c7b4e-4e4f-4fe6-b868-c3f9d300bd99 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:47.086503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a15c7b4e-4e4f-4fe6-b868-c3f9d300bd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:47.150598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 562/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:47.417414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81441eb0-e0ed-4b61-8607-4a4f518df472 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:47.446262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:47.518599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81441eb0-e0ed-4b61-8607-4a4f518df472 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:47.551465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023786f1-0484-45b7-a856-e9d13e7086a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:47.552082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023786f1-0484-45b7-a856-e9d13e7086a8 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.552362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023786f1-0484-45b7-a856-e9d13e7086a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.552631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023786f1-0484-45b7-a856-e9d13e7086a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.553733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023786f1-0484-45b7-a856-e9d13e7086a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:47.555180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023786f1-0484-45b7-a856-e9d13e7086a8 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:47.556718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 571/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:47.562697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c7b4e-4e4f-4fe6-b868-c3f9d300bd99 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:47.608884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:47.657321 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c7b4e-4e4f-4fe6-b868-c3f9d300bd99 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:47.690424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb60d88-1258-4c62-a62c-7b49d0e9695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:47.691568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb60d88-1258-4c62-a62c-7b49d0e9695a None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.691848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb60d88-1258-4c62-a62c-7b49d0e9695a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.692114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb60d88-1258-4c62-a62c-7b49d0e9695a None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.692756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb60d88-1258-4c62-a62c-7b49d0e9695a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:47.694156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb60d88-1258-4c62-a62c-7b49d0e9695a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:47.719970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 563/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:47 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:49.864347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc6c7e-3047-485a-b8ce-63cdea422178 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:49.864868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc6c7e-3047-485a-b8ce-63cdea422178 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:49.864868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc6c7e-3047-485a-b8ce-63cdea422178 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:49.866217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 572/1135] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:49.871314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24d7b-cc61-4c95-99c4-039dcbcb9d8e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:49.871705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24d7b-cc61-4c95-99c4-039dcbcb9d8e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:49.871866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24d7b-cc61-4c95-99c4-039dcbcb9d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:49.891946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d6d24d7b-cc61-4c95-99c4-039dcbcb9d8e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:49.893131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d6d24d7b-cc61-4c95-99c4-039dcbcb9d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:49.959710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 564/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:50.228783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc6c7e-3047-485a-b8ce-63cdea422178 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:50.318303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:50.325666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc6c7e-3047-485a-b8ce-63cdea422178 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:50.359388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b503c-bed8-4d31-a6b7-ef377c8ef7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:50.359996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b503c-bed8-4d31-a6b7-ef377c8ef7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:50.360270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b503c-bed8-4d31-a6b7-ef377c8ef7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:50.360536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b503c-bed8-4d31-a6b7-ef377c8ef7e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:50.361615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b503c-bed8-4d31-a6b7-ef377c8ef7e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:50.363070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b503c-bed8-4d31-a6b7-ef377c8ef7e6 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:50.364587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 573/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:50.370220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d24d7b-cc61-4c95-99c4-039dcbcb9d8e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:50.399313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:50.467668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d24d7b-cc61-4c95-99c4-039dcbcb9d8e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:50.500039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaac7b0-336f-4fd4-add7-f75931a87956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:50.500570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac7b0-336f-4fd4-add7-f75931a87956 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:50.500844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac7b0-336f-4fd4-add7-f75931a87956 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:50.501111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac7b0-336f-4fd4-add7-f75931a87956 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:50.501754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaac7b0-336f-4fd4-add7-f75931a87956 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:50.503690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaac7b0-336f-4fd4-add7-f75931a87956 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:50.524352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 565/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:47:50 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:47:52.678530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194443-74e2-4522-9bd5-fab90fd6c6e2 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.679048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194443-74e2-4522-9bd5-fab90fd6c6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.679048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194443-74e2-4522-9bd5-fab90fd6c6e2 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.680377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 574/1139] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:52.685495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567bdaa-37ff-4de2-aef5-6b05d2fe3b70 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.685878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567bdaa-37ff-4de2-aef5-6b05d2fe3b70 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.686045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567bdaa-37ff-4de2-aef5-6b05d2fe3b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.705059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5567bdaa-37ff-4de2-aef5-6b05d2fe3b70 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:52.706848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5567bdaa-37ff-4de2-aef5-6b05d2fe3b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:52.770269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 566/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:53.037186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06194443-74e2-4522-9bd5-fab90fd6c6e2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.138518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06194443-74e2-4522-9bd5-fab90fd6c6e2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.171717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b074b39-5736-4bcb-8ce4-3aa9284b21dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.172334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b074b39-5736-4bcb-8ce4-3aa9284b21dc None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.172612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b074b39-5736-4bcb-8ce4-3aa9284b21dc None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.172882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b074b39-5736-4bcb-8ce4-3aa9284b21dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.182544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b074b39-5736-4bcb-8ce4-3aa9284b21dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-4b074b39-5736-4bcb-8ce4-3aa9284b21dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:53.187500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:53.188557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:53.189411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 575/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:53.194561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567bdaa-37ff-4de2-aef5-6b05d2fe3b70 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.211901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.291125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567bdaa-37ff-4de2-aef5-6b05d2fe3b70 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.323151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.323742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.324019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.324287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.324940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:53.325238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.326581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.334711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 567/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:53.340521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b074b39-5736-4bcb-8ce4-3aa9284b21dc None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.365025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.437776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b074b39-5736-4bcb-8ce4-3aa9284b21dc None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.471167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478054e-16e9-43fb-abca-9793815b87a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.471799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478054e-16e9-43fb-abca-9793815b87a5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.472076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478054e-16e9-43fb-abca-9793815b87a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.472343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478054e-16e9-43fb-abca-9793815b87a5 None admin] PATH_INFO: `/v3/users/e5608c1ef9434b46b48dee679307e247` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.485477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478054e-16e9-43fb-abca-9793815b87a5 None admin] RBAC: Authorizing `identity:get_user(user_id=e5608c1ef9434b46b48dee679307e247)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.486883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478054e-16e9-43fb-abca-9793815b87a5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.488495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 576/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/users/e5608c1ef9434b46b48dee679307e247 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:53.494278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.571093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.592317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b08b9-1f42-4b7a-b0e0-9a02d6fe139f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.623603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b779a3-d655-43aa-86e8-0e6e585c6ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.624192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b779a3-d655-43aa-86e8-0e6e585c6ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.624469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b779a3-d655-43aa-86e8-0e6e585c6ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.624738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b779a3-d655-43aa-86e8-0e6e585c6ce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.688321 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b779a3-d655-43aa-86e8-0e6e585c6ce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.689729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b779a3-d655-43aa-86e8-0e6e585c6ce1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.763179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.784332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 568/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:53.790291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478054e-16e9-43fb-abca-9793815b87a5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.867325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.886123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478054e-16e9-43fb-abca-9793815b87a5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.917630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a86855-37d1-4b35-b013-4d6d458ff459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.918236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a86855-37d1-4b35-b013-4d6d458ff459 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.918509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a86855-37d1-4b35-b013-4d6d458ff459 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.918776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a86855-37d1-4b35-b013-4d6d458ff459 None admin] PATH_INFO: `/v3/projects/d0b649a03ce64ae69915eddb448da3c6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.919882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a86855-37d1-4b35-b013-4d6d458ff459 None admin] RBAC: Authorizing `identity:get_project(project_id=d0b649a03ce64ae69915eddb448da3c6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.921286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a86855-37d1-4b35-b013-4d6d458ff459 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.922858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 577/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/projects/d0b649a03ce64ae69915eddb448da3c6 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:53.929382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b779a3-d655-43aa-86e8-0e6e585c6ce1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:54.024658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b779a3-d655-43aa-86e8-0e6e585c6ce1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:54.057441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a45e7-81c1-4c63-b82a-e626e81bb4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:54.058022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a45e7-81c1-4c63-b82a-e626e81bb4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.058300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a45e7-81c1-4c63-b82a-e626e81bb4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.058566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a45e7-81c1-4c63-b82a-e626e81bb4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.059307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a45e7-81c1-4c63-b82a-e626e81bb4f4 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=d0b649a03ce64ae69915eddb448da3c6, user.id=e5608c1ef9434b46b48dee679307e247) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:54.059609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a45e7-81c1-4c63-b82a-e626e81bb4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:54.061002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a45e7-81c1-4c63-b82a-e626e81bb4f4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:54.066929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 569/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/role_assignments?user.id=e5608c1ef9434b46b48dee679307e247&scope.project.id=d0b649a03ce64ae69915eddb448da3c6&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:56.147533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf0758-18ad-4571-8ae1-51b5ae116eae None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:56.148087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf0758-18ad-4571-8ae1-51b5ae116eae None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:56.148087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf0758-18ad-4571-8ae1-51b5ae116eae None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:56.149406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 578/1147] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:56.154363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0349a-d1f7-4731-ac0e-31a64b5892f1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:56.154795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0349a-d1f7-4731-ac0e-31a64b5892f1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:56.154930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0349a-d1f7-4731-ac0e-31a64b5892f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:56.172561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a7d0349a-d1f7-4731-ac0e-31a64b5892f1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:56.173736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a7d0349a-d1f7-4731-ac0e-31a64b5892f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:56.208102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:56.238114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 570/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:56.503328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf0758-18ad-4571-8ae1-51b5ae116eae None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:56.601289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf0758-18ad-4571-8ae1-51b5ae116eae None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:56.634080 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9117f-7aa4-48aa-83eb-5a2637359425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:56.634694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9117f-7aa4-48aa-83eb-5a2637359425 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:56.635001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9117f-7aa4-48aa-83eb-5a2637359425 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:56.635276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9117f-7aa4-48aa-83eb-5a2637359425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:56.644995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9117f-7aa4-48aa-83eb-5a2637359425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-69d9117f-7aa4-48aa-83eb-5a2637359425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:56.649707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:56.650771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:47:56.651847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 579/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:56.656609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0349a-d1f7-4731-ac0e-31a64b5892f1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:56.752275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0349a-d1f7-4731-ac0e-31a64b5892f1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:56.784902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:56.785495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:56.785774 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:56.786046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:56.786696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:56.787026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:56.788381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:56.796078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 571/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:56.801967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9117f-7aa4-48aa-83eb-5a2637359425 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:56.813131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:56.898501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9117f-7aa4-48aa-83eb-5a2637359425 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:56.931084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878d025-1c50-4b80-aa68-4eddd5086074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:56.931643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878d025-1c50-4b80-aa68-4eddd5086074 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:56.931922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878d025-1c50-4b80-aa68-4eddd5086074 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:56.932193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878d025-1c50-4b80-aa68-4eddd5086074 None admin] PATH_INFO: `/v3/users/e5608c1ef9434b46b48dee679307e247` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:56.933296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878d025-1c50-4b80-aa68-4eddd5086074 None admin] RBAC: Authorizing `identity:get_user(user_id=e5608c1ef9434b46b48dee679307e247)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:56.934725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878d025-1c50-4b80-aa68-4eddd5086074 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:56.936557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 580/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:47:56 2022] GET /identity/v3/users/e5608c1ef9434b46b48dee679307e247 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:56.942741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:56.975525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:57.040158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52ea89-d106-4b05-a654-cfe448da7433 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:57.072867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cee07d-c9c0-4166-9b76-0bf52bea41c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:57.073481 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cee07d-c9c0-4166-9b76-0bf52bea41c5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:57.073757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cee07d-c9c0-4166-9b76-0bf52bea41c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:57.074024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cee07d-c9c0-4166-9b76-0bf52bea41c5 None admin] PATH_INFO: `/v3/projects/d0b649a03ce64ae69915eddb448da3c6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:57.075147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cee07d-c9c0-4166-9b76-0bf52bea41c5 None admin] RBAC: Authorizing `identity:get_project(project_id=d0b649a03ce64ae69915eddb448da3c6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:57.076565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cee07d-c9c0-4166-9b76-0bf52bea41c5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:57.078169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 572/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:47:56 2022] GET /identity/v3/projects/d0b649a03ce64ae69915eddb448da3c6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:57.084065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878d025-1c50-4b80-aa68-4eddd5086074 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:57.159776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:57.182968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878d025-1c50-4b80-aa68-4eddd5086074 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:57.214687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647af14-ef12-4fb3-9d7a-00b888bd3d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:57.215350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647af14-ef12-4fb3-9d7a-00b888bd3d13 None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:57.215629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647af14-ef12-4fb3-9d7a-00b888bd3d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:57.215901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647af14-ef12-4fb3-9d7a-00b888bd3d13 None admin] PATH_INFO: `/v3/projects/d0b649a03ce64ae69915eddb448da3c6/users/e5608c1ef9434b46b48dee679307e247/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:57.217760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647af14-ef12-4fb3-9d7a-00b888bd3d13 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b649a03ce64ae69915eddb448da3c6, user_id=e5608c1ef9434b46b48dee679307e247, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:57.219202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647af14-ef12-4fb3-9d7a-00b888bd3d13 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:57.232331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 581/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:57 2022] PUT /identity/v3/projects/d0b649a03ce64ae69915eddb448da3c6/users/e5608c1ef9434b46b48dee679307e247/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:47:59.358083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662b4e-05f6-4d34-958a-d5274f858ab3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.358668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662b4e-05f6-4d34-958a-d5274f858ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.358668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662b4e-05f6-4d34-958a-d5274f858ab3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.359997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 573/1154] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:59.365701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0613695-7b17-46c9-b789-fe57e11ed27a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.366136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0613695-7b17-46c9-b789-fe57e11ed27a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.366233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0613695-7b17-46c9-b789-fe57e11ed27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.383889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c0613695-7b17-46c9-b789-fe57e11ed27a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:59.385171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c0613695-7b17-46c9-b789-fe57e11ed27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:59.449792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 582/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:59.719101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662b4e-05f6-4d34-958a-d5274f858ab3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.819048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662b4e-05f6-4d34-958a-d5274f858ab3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:59.820563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:59.853532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505dd96-b02d-41e1-bbf5-99e3ad307a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:59.854152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505dd96-b02d-41e1-bbf5-99e3ad307a2c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.854431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505dd96-b02d-41e1-bbf5-99e3ad307a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.854705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505dd96-b02d-41e1-bbf5-99e3ad307a2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.864451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505dd96-b02d-41e1-bbf5-99e3ad307a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2505dd96-b02d-41e1-bbf5-99e3ad307a2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:59.869187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:59.870319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:47:59.871470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 574/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:59.876322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0613695-7b17-46c9-b789-fe57e11ed27a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.938721 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:59.972908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0613695-7b17-46c9-b789-fe57e11ed27a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.004682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.005271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.005544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.005811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.006455 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:00.006748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.008126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.015887 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 583/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:00.022263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505dd96-b02d-41e1-bbf5-99e3ad307a2c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.120778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505dd96-b02d-41e1-bbf5-99e3ad307a2c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.153618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f6415-5e20-4626-a57c-cb7dfc1bdea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.154230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6415-5e20-4626-a57c-cb7dfc1bdea0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.154507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6415-5e20-4626-a57c-cb7dfc1bdea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.154776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6415-5e20-4626-a57c-cb7dfc1bdea0 None admin] PATH_INFO: `/v3/users/e5608c1ef9434b46b48dee679307e247` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.155916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f6415-5e20-4626-a57c-cb7dfc1bdea0 None admin] RBAC: Authorizing `identity:get_user(user_id=e5608c1ef9434b46b48dee679307e247)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.157327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f6415-5e20-4626-a57c-cb7dfc1bdea0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.158877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 575/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/users/e5608c1ef9434b46b48dee679307e247 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:00.164983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.261148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804c9a5-84c1-4225-8aa5-f50926c6bd4a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.286078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:00.293912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f14476-c2b7-42a0-ba0a-cacfcbda2265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.294504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f14476-c2b7-42a0-ba0a-cacfcbda2265 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.294783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f14476-c2b7-42a0-ba0a-cacfcbda2265 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.295081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f14476-c2b7-42a0-ba0a-cacfcbda2265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.359482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f14476-c2b7-42a0-ba0a-cacfcbda2265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.360899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f14476-c2b7-42a0-ba0a-cacfcbda2265 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.455362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 584/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:00.461841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f6415-5e20-4626-a57c-cb7dfc1bdea0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.491353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:00.558449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f6415-5e20-4626-a57c-cb7dfc1bdea0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.591249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65bf4f-384f-4822-9b38-5bd933835067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.591861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65bf4f-384f-4822-9b38-5bd933835067 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.592139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65bf4f-384f-4822-9b38-5bd933835067 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.592407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65bf4f-384f-4822-9b38-5bd933835067 None admin] PATH_INFO: `/v3/projects/d0b649a03ce64ae69915eddb448da3c6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.593499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65bf4f-384f-4822-9b38-5bd933835067 None admin] RBAC: Authorizing `identity:get_project(project_id=d0b649a03ce64ae69915eddb448da3c6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.594923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65bf4f-384f-4822-9b38-5bd933835067 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.596527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 576/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/projects/d0b649a03ce64ae69915eddb448da3c6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:00.602673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f14476-c2b7-42a0-ba0a-cacfcbda2265 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.629560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:00.697343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f14476-c2b7-42a0-ba0a-cacfcbda2265 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.728857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215dbee-2083-481b-a336-1db3ba8e26c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.729443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215dbee-2083-481b-a336-1db3ba8e26c0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.729718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215dbee-2083-481b-a336-1db3ba8e26c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.729987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215dbee-2083-481b-a336-1db3ba8e26c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.730695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215dbee-2083-481b-a336-1db3ba8e26c0 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=d0b649a03ce64ae69915eddb448da3c6, user.id=e5608c1ef9434b46b48dee679307e247) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:00.731025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215dbee-2083-481b-a336-1db3ba8e26c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.732458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215dbee-2083-481b-a336-1db3ba8e26c0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.738374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 585/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/role_assignments?user.id=e5608c1ef9434b46b48dee679307e247&scope.project.id=d0b649a03ce64ae69915eddb448da3c6&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:02.895485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0020afe-d05a-4537-92a4-54cb65ff94f6 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:02.896023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0020afe-d05a-4537-92a4-54cb65ff94f6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:02.896023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0020afe-d05a-4537-92a4-54cb65ff94f6 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:02.897352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 577/1162] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:02.902430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963da6a3-0464-496a-a1fe-45f65a52b9bd None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:02.902851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963da6a3-0464-496a-a1fe-45f65a52b9bd None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:02.902989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963da6a3-0464-496a-a1fe-45f65a52b9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:02.921027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-963da6a3-0464-496a-a1fe-45f65a52b9bd None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:02.922183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-963da6a3-0464-496a-a1fe-45f65a52b9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:02.984981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 586/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:03.249596 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0020afe-d05a-4537-92a4-54cb65ff94f6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:03.342981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:03.348630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0020afe-d05a-4537-92a4-54cb65ff94f6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:03.381517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c13d5-6967-45d3-8d23-967b7185c04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:03.382105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c13d5-6967-45d3-8d23-967b7185c04f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:03.382385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c13d5-6967-45d3-8d23-967b7185c04f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:03.382653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c13d5-6967-45d3-8d23-967b7185c04f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:03.446794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c13d5-6967-45d3-8d23-967b7185c04f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:03.448250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c13d5-6967-45d3-8d23-967b7185c04f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:03.537353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:03.542042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 578/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:03.547963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963da6a3-0464-496a-a1fe-45f65a52b9bd None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:03.571766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:03.643957 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963da6a3-0464-496a-a1fe-45f65a52b9bd None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:03.677121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20472bf1-4318-461f-8896-4cc1f98dd540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:03.677732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20472bf1-4318-461f-8896-4cc1f98dd540 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:03.678010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20472bf1-4318-461f-8896-4cc1f98dd540 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:03.678279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20472bf1-4318-461f-8896-4cc1f98dd540 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:03.684689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20472bf1-4318-461f-8896-4cc1f98dd540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:03.686360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20472bf1-4318-461f-8896-4cc1f98dd540 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-20472bf1-4318-461f-8896-4cc1f98dd540 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:03.697886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:03.699108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:48:03.700138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 587/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:48:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:03.704865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c13d5-6967-45d3-8d23-967b7185c04f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:03.799963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c13d5-6967-45d3-8d23-967b7185c04f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:03.832806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3bcac-3e3a-4a97-a4e8-b6ea36be4c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:03.833395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3bcac-3e3a-4a97-a4e8-b6ea36be4c56 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:03.833677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3bcac-3e3a-4a97-a4e8-b6ea36be4c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:03.833946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3bcac-3e3a-4a97-a4e8-b6ea36be4c56 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:03.834576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3bcac-3e3a-4a97-a4e8-b6ea36be4c56 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:03.834877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3bcac-3e3a-4a97-a4e8-b6ea36be4c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:03.836241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3bcac-3e3a-4a97-a4e8-b6ea36be4c56 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:03.843060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 579/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Mon Oct 10 11:48:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:05.939785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e4635-d382-446d-82eb-87ea59a6a7cf None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.940946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e4635-d382-446d-82eb-87ea59a6a7cf None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.940946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e4635-d382-446d-82eb-87ea59a6a7cf None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.942281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 588/1167] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:05.947618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a389-5272-4d36-bf35-463fdfff1f31 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.948029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a389-5272-4d36-bf35-463fdfff1f31 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.948161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a389-5272-4d36-bf35-463fdfff1f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.966334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6ba0a389-5272-4d36-bf35-463fdfff1f31 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:05.967525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6ba0a389-5272-4d36-bf35-463fdfff1f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:06.021871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:06.032822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 580/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:06.300218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e4635-d382-446d-82eb-87ea59a6a7cf None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:06.379904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:06.399190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e4635-d382-446d-82eb-87ea59a6a7cf None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:06.431552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec9980-7600-42c1-81bf-967b8ca29575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:06.432150 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec9980-7600-42c1-81bf-967b8ca29575 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:06.432428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec9980-7600-42c1-81bf-967b8ca29575 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:06.432695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec9980-7600-42c1-81bf-967b8ca29575 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:06.433342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec9980-7600-42c1-81bf-967b8ca29575 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:06.434684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec9980-7600-42c1-81bf-967b8ca29575 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:06.451005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 589/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Mon Oct 10 11:48:06 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:48:08.587163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526419e5-ed60-4c57-bcbc-387c6c3bd731 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:08.587710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526419e5-ed60-4c57-bcbc-387c6c3bd731 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:08.587710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526419e5-ed60-4c57-bcbc-387c6c3bd731 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:08.589017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 581/1170] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:08.594239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d263-3dcc-4b0d-8e0e-c7e5be6f51b2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:08.594656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d263-3dcc-4b0d-8e0e-c7e5be6f51b2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:08.594766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d263-3dcc-4b0d-8e0e-c7e5be6f51b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:08.612403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-67d4d263-3dcc-4b0d-8e0e-c7e5be6f51b2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:08.613568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-67d4d263-3dcc-4b0d-8e0e-c7e5be6f51b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:08.676445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 590/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:08.945978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526419e5-ed60-4c57-bcbc-387c6c3bd731 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:09.043109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526419e5-ed60-4c57-bcbc-387c6c3bd731 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:09.075265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ca02b-74b0-4285-b59c-cced2b088e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:09.075881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ca02b-74b0-4285-b59c-cced2b088e35 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:09.076161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ca02b-74b0-4285-b59c-cced2b088e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:09.076462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ca02b-74b0-4285-b59c-cced2b088e35 None admin] PATH_INFO: `/v3/domains/7634400e65454facb3cb779f7bdbcd5a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:09.087394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ca02b-74b0-4285-b59c-cced2b088e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7634400e65454facb3cb779f7bdbcd5a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:09.088821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ca02b-74b0-4285-b59c-cced2b088e35 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:09.090381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 582/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Mon Oct 10 11:48:08 2022] GET /identity/v3/domains/7634400e65454facb3cb779f7bdbcd5a => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:09.096801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d263-3dcc-4b0d-8e0e-c7e5be6f51b2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:09.162908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:09.192457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d263-3dcc-4b0d-8e0e-c7e5be6f51b2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:09.224768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c41e4-99ce-4e45-9787-53ebab7a8061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:09.225309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c41e4-99ce-4e45-9787-53ebab7a8061 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:09.225585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c41e4-99ce-4e45-9787-53ebab7a8061 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:09.225850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c41e4-99ce-4e45-9787-53ebab7a8061 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:09.226478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c41e4-99ce-4e45-9787-53ebab7a8061 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:09.227920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c41e4-99ce-4e45-9787-53ebab7a8061 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:09.252916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 591/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:48:09 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:48:11.395353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b350b0-5645-41bf-b698-b86b9343fd59 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:11.395917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b350b0-5645-41bf-b698-b86b9343fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:11.395917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b350b0-5645-41bf-b698-b86b9343fd59 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:11.397237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 583/1174] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:11.402380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba011e-24ae-490d-9e1c-823cc6817411 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:11.402800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba011e-24ae-490d-9e1c-823cc6817411 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:11.402965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba011e-24ae-490d-9e1c-823cc6817411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:11.420724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f2ba011e-24ae-490d-9e1c-823cc6817411 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:11.421899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f2ba011e-24ae-490d-9e1c-823cc6817411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:11.485756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 592/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:11.750412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b350b0-5645-41bf-b698-b86b9343fd59 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:11.802905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:11.849358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b350b0-5645-41bf-b698-b86b9343fd59 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:11.881824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d1208-6d0a-4a33-bfde-f2d0892ca7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:11.882427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d1208-6d0a-4a33-bfde-f2d0892ca7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:11.882701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d1208-6d0a-4a33-bfde-f2d0892ca7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:11.882995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d1208-6d0a-4a33-bfde-f2d0892ca7a8 None admin] PATH_INFO: `/v3/domains/7634400e65454facb3cb779f7bdbcd5a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:11.884091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d1208-6d0a-4a33-bfde-f2d0892ca7a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7634400e65454facb3cb779f7bdbcd5a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:11.885493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d1208-6d0a-4a33-bfde-f2d0892ca7a8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:11.887028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 584/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Mon Oct 10 11:48:11 2022] GET /identity/v3/domains/7634400e65454facb3cb779f7bdbcd5a => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:11.892576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba011e-24ae-490d-9e1c-823cc6817411 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:11.946317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:11.989843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba011e-24ae-490d-9e1c-823cc6817411 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:12.022615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fe473-cc7f-4362-8210-27f0304a5d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:12.023175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fe473-cc7f-4362-8210-27f0304a5d9c None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:12.023451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fe473-cc7f-4362-8210-27f0304a5d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:12.023717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fe473-cc7f-4362-8210-27f0304a5d9c None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:12.024355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fe473-cc7f-4362-8210-27f0304a5d9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:12.025760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fe473-cc7f-4362-8210-27f0304a5d9c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:12.046982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 593/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:48:11 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:48:14.210762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f128-37c3-442e-a72e-5230b49b10f0 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.211312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f128-37c3-442e-a72e-5230b49b10f0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.211312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f128-37c3-442e-a72e-5230b49b10f0 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:14.212656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 585/1178] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:14.217774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a59a6-4493-45f0-95ae-70149a52a3e7 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.218203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a59a6-4493-45f0-95ae-70149a52a3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.218307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a59a6-4493-45f0-95ae-70149a52a3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:14.236701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e03a59a6-4493-45f0-95ae-70149a52a3e7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:14.237892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e03a59a6-4493-45f0-95ae-70149a52a3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:14.302308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 594/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:14.570573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172f128-37c3-442e-a72e-5230b49b10f0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:14.667647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172f128-37c3-442e-a72e-5230b49b10f0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:14.686593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:14.700796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266dc37-1a9f-4943-91fd-a2046e7a1485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:14.701403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266dc37-1a9f-4943-91fd-a2046e7a1485 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.701677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266dc37-1a9f-4943-91fd-a2046e7a1485 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.701946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266dc37-1a9f-4943-91fd-a2046e7a1485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:14.712148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266dc37-1a9f-4943-91fd-a2046e7a1485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1266dc37-1a9f-4943-91fd-a2046e7a1485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:14.716792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:14.717864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:48:14.718830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 586/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:14.723727 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a59a6-4493-45f0-95ae-70149a52a3e7 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:14.764036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:14.820666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a59a6-4493-45f0-95ae-70149a52a3e7 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:14.852311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:14.852908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.853193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.853463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:14.854117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:14.854420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:14.855809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:14.863999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 595/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:14.869876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266dc37-1a9f-4943-91fd-a2046e7a1485 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:14.966346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266dc37-1a9f-4943-91fd-a2046e7a1485 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:14.998513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bcbcc-f029-43ba-bc21-8ebeae574cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:14.999136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bcbcc-f029-43ba-bc21-8ebeae574cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.999418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bcbcc-f029-43ba-bc21-8ebeae574cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.999685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bcbcc-f029-43ba-bc21-8ebeae574cf4 None admin] PATH_INFO: `/v3/users/3806de8459e54e7ba06805d93ac357a1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.013178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bcbcc-f029-43ba-bc21-8ebeae574cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=3806de8459e54e7ba06805d93ac357a1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.014578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bcbcc-f029-43ba-bc21-8ebeae574cf4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.016137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 587/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:14 2022] GET /identity/v3/users/3806de8459e54e7ba06805d93ac357a1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:15.021766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.118567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef32bc-b454-4146-91b8-f7a8ba300557 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.120082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.153142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f04a8-3be0-4b56-afa5-85f15267d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.153737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f04a8-3be0-4b56-afa5-85f15267d33e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.154016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f04a8-3be0-4b56-afa5-85f15267d33e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.154286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f04a8-3be0-4b56-afa5-85f15267d33e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.218549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f04a8-3be0-4b56-afa5-85f15267d33e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.219993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f04a8-3be0-4b56-afa5-85f15267d33e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.314115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 596/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:15.320515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bcbcc-f029-43ba-bc21-8ebeae574cf4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.359293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.418682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bcbcc-f029-43ba-bc21-8ebeae574cf4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.451906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d029c4-b352-4a6f-b470-0eacb4c573be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.452525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d029c4-b352-4a6f-b470-0eacb4c573be None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.452803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d029c4-b352-4a6f-b470-0eacb4c573be None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.453075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d029c4-b352-4a6f-b470-0eacb4c573be None admin] PATH_INFO: `/v3/projects/8715d8ce5fe6446fb74bf9b7a4521693` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.454182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d029c4-b352-4a6f-b470-0eacb4c573be None admin] RBAC: Authorizing `identity:get_project(project_id=8715d8ce5fe6446fb74bf9b7a4521693)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.456246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d029c4-b352-4a6f-b470-0eacb4c573be None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.457852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 588/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/projects/8715d8ce5fe6446fb74bf9b7a4521693 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:15.463853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f04a8-3be0-4b56-afa5-85f15267d33e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.465971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.560374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f04a8-3be0-4b56-afa5-85f15267d33e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.594037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb4dcb-6a01-4509-9d49-2d2fddb4f610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.594623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb4dcb-6a01-4509-9d49-2d2fddb4f610 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.594928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb4dcb-6a01-4509-9d49-2d2fddb4f610 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.595206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb4dcb-6a01-4509-9d49-2d2fddb4f610 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.595938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb4dcb-6a01-4509-9d49-2d2fddb4f610 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=8715d8ce5fe6446fb74bf9b7a4521693, user.id=3806de8459e54e7ba06805d93ac357a1) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:15.596250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb4dcb-6a01-4509-9d49-2d2fddb4f610 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.597669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb4dcb-6a01-4509-9d49-2d2fddb4f610 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.602776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 597/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/role_assignments?user.id=3806de8459e54e7ba06805d93ac357a1&scope.project.id=8715d8ce5fe6446fb74bf9b7a4521693&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:17.711169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d56da-3ce7-43b8-92ac-01d51a6e0ded None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:17.711682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d56da-3ce7-43b8-92ac-01d51a6e0ded None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:17.711682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d56da-3ce7-43b8-92ac-01d51a6e0ded None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:17.712979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 589/1186] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:17.718034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f753f7-9aee-411e-a1bf-ea648ca5f0d1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:17.718398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f753f7-9aee-411e-a1bf-ea648ca5f0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:17.718593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f753f7-9aee-411e-a1bf-ea648ca5f0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:17.736253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-88f753f7-9aee-411e-a1bf-ea648ca5f0d1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:17.738166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-88f753f7-9aee-411e-a1bf-ea648ca5f0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:17.794759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:17.802683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 598/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:18.073138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d56da-3ce7-43b8-92ac-01d51a6e0ded None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:18.171566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d56da-3ce7-43b8-92ac-01d51a6e0ded None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:18.178544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:18.204788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa741e-d39d-4557-b626-36d86d5d4073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:18.205990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa741e-d39d-4557-b626-36d86d5d4073 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:18.206271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa741e-d39d-4557-b626-36d86d5d4073 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:18.206543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa741e-d39d-4557-b626-36d86d5d4073 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:18.216249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa741e-d39d-4557-b626-36d86d5d4073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d7aa741e-d39d-4557-b626-36d86d5d4073 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:18.220990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:18.222070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:48:18.223129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 590/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:18.227918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f753f7-9aee-411e-a1bf-ea648ca5f0d1 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:18.328172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f753f7-9aee-411e-a1bf-ea648ca5f0d1 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:18.360729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:18.361327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:18.361739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:18.362012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:18.362664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:18.362995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:18.364356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:18.372054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 599/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:18.378529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa741e-d39d-4557-b626-36d86d5d4073 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:18.474867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa741e-d39d-4557-b626-36d86d5d4073 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:18.507891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06b709-d574-437b-afac-7e760eba4f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:18.508502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06b709-d574-437b-afac-7e760eba4f0d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:18.508778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06b709-d574-437b-afac-7e760eba4f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:18.509048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06b709-d574-437b-afac-7e760eba4f0d None admin] PATH_INFO: `/v3/users/3806de8459e54e7ba06805d93ac357a1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:18.510145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06b709-d574-437b-afac-7e760eba4f0d None admin] RBAC: Authorizing `identity:get_user(user_id=3806de8459e54e7ba06805d93ac357a1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:18.511595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06b709-d574-437b-afac-7e760eba4f0d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:18.513139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 591/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:18 2022] GET /identity/v3/users/3806de8459e54e7ba06805d93ac357a1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:18.518877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:18.575508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:18.616348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ebec8-26fc-48ff-b202-c2ec3dc6f248 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:18.648493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bab3d9-9208-4c2b-afd7-d4c8ec5248a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:18.649104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bab3d9-9208-4c2b-afd7-d4c8ec5248a1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:18.649385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bab3d9-9208-4c2b-afd7-d4c8ec5248a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:18.649658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bab3d9-9208-4c2b-afd7-d4c8ec5248a1 None admin] PATH_INFO: `/v3/projects/8715d8ce5fe6446fb74bf9b7a4521693` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:18.650760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bab3d9-9208-4c2b-afd7-d4c8ec5248a1 None admin] RBAC: Authorizing `identity:get_project(project_id=8715d8ce5fe6446fb74bf9b7a4521693)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:18.652204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bab3d9-9208-4c2b-afd7-d4c8ec5248a1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:18.653795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 600/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:18 2022] GET /identity/v3/projects/8715d8ce5fe6446fb74bf9b7a4521693 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:18.659585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06b709-d574-437b-afac-7e760eba4f0d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:18.693130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:18.755705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06b709-d574-437b-afac-7e760eba4f0d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:18.787491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f101edd-2359-4c7f-bcaf-6525cecd45c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:18.788118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101edd-2359-4c7f-bcaf-6525cecd45c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:18.788394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101edd-2359-4c7f-bcaf-6525cecd45c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:18.788659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f101edd-2359-4c7f-bcaf-6525cecd45c9 None admin] PATH_INFO: `/v3/projects/8715d8ce5fe6446fb74bf9b7a4521693/users/3806de8459e54e7ba06805d93ac357a1/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:18.790503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f101edd-2359-4c7f-bcaf-6525cecd45c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8715d8ce5fe6446fb74bf9b7a4521693, user_id=3806de8459e54e7ba06805d93ac357a1, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:18.791926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f101edd-2359-4c7f-bcaf-6525cecd45c9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:18.804270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 592/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:48:18 2022] PUT /identity/v3/projects/8715d8ce5fe6446fb74bf9b7a4521693/users/3806de8459e54e7ba06805d93ac357a1/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:48:20.925745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81be56-7326-4311-b99f-c34cfc2882ab None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.926281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81be56-7326-4311-b99f-c34cfc2882ab None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.926281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81be56-7326-4311-b99f-c34cfc2882ab None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.927654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 601/1193] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:20.932855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693411-3f83-4791-a47b-c356c918dc8e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.933242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693411-3f83-4791-a47b-c356c918dc8e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.933398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693411-3f83-4791-a47b-c356c918dc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.951997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c1693411-3f83-4791-a47b-c356c918dc8e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:20.953172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c1693411-3f83-4791-a47b-c356c918dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:21.016712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 593/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:21.284903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81be56-7326-4311-b99f-c34cfc2882ab None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.383659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81be56-7326-4311-b99f-c34cfc2882ab None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.407867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.417195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb125-637b-4e88-92eb-69f7ed0207fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.417818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb125-637b-4e88-92eb-69f7ed0207fb None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.418096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb125-637b-4e88-92eb-69f7ed0207fb None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.418364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb125-637b-4e88-92eb-69f7ed0207fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.428646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcb125-637b-4e88-92eb-69f7ed0207fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-f5dcb125-637b-4e88-92eb-69f7ed0207fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:21.433329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:21.434416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:48:21.435790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 602/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:21.440657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693411-3f83-4791-a47b-c356c918dc8e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.464238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.537433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693411-3f83-4791-a47b-c356c918dc8e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.569112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.569696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.569969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.570234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.570877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:21.571200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.572531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.580639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 594/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:21.586404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb125-637b-4e88-92eb-69f7ed0207fb None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.681478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb125-637b-4e88-92eb-69f7ed0207fb None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.714581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27a61-0345-4c4b-8ed1-3dd2362bff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.715210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27a61-0345-4c4b-8ed1-3dd2362bff5d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.715486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27a61-0345-4c4b-8ed1-3dd2362bff5d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.715757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27a61-0345-4c4b-8ed1-3dd2362bff5d None admin] PATH_INFO: `/v3/users/3806de8459e54e7ba06805d93ac357a1` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.716846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe27a61-0345-4c4b-8ed1-3dd2362bff5d None admin] RBAC: Authorizing `identity:get_user(user_id=3806de8459e54e7ba06805d93ac357a1)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.718251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe27a61-0345-4c4b-8ed1-3dd2362bff5d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.719797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 603/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/users/3806de8459e54e7ba06805d93ac357a1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:21.725953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.806223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.820669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693313bc-9f7e-46fe-bd5a-3c217cad450c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.853129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7a16-1d56-4e4a-80de-8ce53ca82657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.853716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7a16-1d56-4e4a-80de-8ce53ca82657 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.853996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7a16-1d56-4e4a-80de-8ce53ca82657 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.854266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f7a16-1d56-4e4a-80de-8ce53ca82657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.917850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f7a16-1d56-4e4a-80de-8ce53ca82657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.919294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f7a16-1d56-4e4a-80de-8ce53ca82657 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.997600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:22.013054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 595/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:22.019436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27a61-0345-4c4b-8ed1-3dd2362bff5d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:22.072192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:22.115386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27a61-0345-4c4b-8ed1-3dd2362bff5d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:22.148144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b596a-b810-427a-9774-469ec2139faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:22.148751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b596a-b810-427a-9774-469ec2139faf None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:22.149025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b596a-b810-427a-9774-469ec2139faf None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:22.149293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b596a-b810-427a-9774-469ec2139faf None admin] PATH_INFO: `/v3/projects/8715d8ce5fe6446fb74bf9b7a4521693` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:22.150377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b596a-b810-427a-9774-469ec2139faf None admin] RBAC: Authorizing `identity:get_project(project_id=8715d8ce5fe6446fb74bf9b7a4521693)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:22.151847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b596a-b810-427a-9774-469ec2139faf None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:22.153432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 604/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:22 2022] GET /identity/v3/projects/8715d8ce5fe6446fb74bf9b7a4521693 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:22.159521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7a16-1d56-4e4a-80de-8ce53ca82657 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:22.258326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f7a16-1d56-4e4a-80de-8ce53ca82657 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:22.291481 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638dd39-ab3b-42ad-be76-91d361892828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:22.292058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638dd39-ab3b-42ad-be76-91d361892828 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:22.292404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638dd39-ab3b-42ad-be76-91d361892828 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:22.292724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638dd39-ab3b-42ad-be76-91d361892828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:22.293554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638dd39-ab3b-42ad-be76-91d361892828 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=8715d8ce5fe6446fb74bf9b7a4521693, user.id=3806de8459e54e7ba06805d93ac357a1) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:22.293872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638dd39-ab3b-42ad-be76-91d361892828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:22.295393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638dd39-ab3b-42ad-be76-91d361892828 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:22.301792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 596/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:22 2022] GET /identity/v3/role_assignments?user.id=3806de8459e54e7ba06805d93ac357a1&scope.project.id=8715d8ce5fe6446fb74bf9b7a4521693&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:32.144205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=39 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:32.298561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=75 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:17.663942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08a96a-52b1-4325-89fd-bf9b942aa1ed None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:17.664497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08a96a-52b1-4325-89fd-bf9b942aa1ed None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:17.664497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08a96a-52b1-4325-89fd-bf9b942aa1ed None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:17.665749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 605/1201] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:17.672451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5e7d-d8b5-4f8c-b1de-2f6064e86a98 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:17.673023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5e7d-d8b5-4f8c-b1de-2f6064e86a98 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:17.673023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5e7d-d8b5-4f8c-b1de-2f6064e86a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:17.694688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-58fe5e7d-d8b5-4f8c-b1de-2f6064e86a98 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:17.695932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-58fe5e7d-d8b5-4f8c-b1de-2f6064e86a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:17.782152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 597/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:18.057773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08a96a-52b1-4325-89fd-bf9b942aa1ed None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:18.185856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08a96a-52b1-4325-89fd-bf9b942aa1ed None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:18.223829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:18.224429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:18.224710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:18.224981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:18.237444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:18.238866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:18.257463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:18.258577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:18.259855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:18.259855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:18.259855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:18.266924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 606/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:18.267064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe5e7d-d8b5-4f8c-b1de-2f6064e86a98 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:18.404596 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe5e7d-d8b5-4f8c-b1de-2f6064e86a98 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:18.441975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:18.449074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a20ff-3727-4e6d-973c-e84eeea3a86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:18.450025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a20ff-3727-4e6d-973c-e84eeea3a86f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:18.450486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a20ff-3727-4e6d-973c-e84eeea3a86f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:18.451027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a20ff-3727-4e6d-973c-e84eeea3a86f None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:18.451867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a20ff-3727-4e6d-973c-e84eeea3a86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:18.452191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a20ff-3727-4e6d-973c-e84eeea3a86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:18.454174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a20ff-3727-4e6d-973c-e84eeea3a86f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:18.467251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 598/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:18.473184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:18.538458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:18.620023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b419-9a8d-4948-9047-6d09b4566c99 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:18.671094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c5f3a-06f3-4f0d-aa7a-74c84e408015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:18.672321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c5f3a-06f3-4f0d-aa7a-74c84e408015 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:18.672601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c5f3a-06f3-4f0d-aa7a-74c84e408015 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:18.672870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c5f3a-06f3-4f0d-aa7a-74c84e408015 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:18.673516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c5f3a-06f3-4f0d-aa7a-74c84e408015 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:18.674967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c5f3a-06f3-4f0d-aa7a-74c84e408015 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:18.701756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 607/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Mon Oct 10 11:49:18 2022] POST /identity/v3/users => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:49:21.065185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ff4db-59a9-4eca-b54a-8a5146a2dffc None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:21.065733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ff4db-59a9-4eca-b54a-8a5146a2dffc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:21.065733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ff4db-59a9-4eca-b54a-8a5146a2dffc None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:21.067104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 599/1206] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:21.072692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764d13-12f4-450e-886d-dc382a4e114f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:21.074469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764d13-12f4-450e-886d-dc382a4e114f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:21.074469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764d13-12f4-450e-886d-dc382a4e114f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:21.094190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c9764d13-12f4-450e-886d-dc382a4e114f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:21.095490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c9764d13-12f4-450e-886d-dc382a4e114f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:21.172106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 608/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:21.440303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ff4db-59a9-4eca-b54a-8a5146a2dffc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:21.559674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ff4db-59a9-4eca-b54a-8a5146a2dffc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:21.599635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebb40d-81bf-42d1-bf48-04fc574c7b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:21.600627 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebb40d-81bf-42d1-bf48-04fc574c7b44 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:21.601106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebb40d-81bf-42d1-bf48-04fc574c7b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:21.601376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebb40d-81bf-42d1-bf48-04fc574c7b44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:21.613017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ebb40d-81bf-42d1-bf48-04fc574c7b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-27ebb40d-81bf-42d1-bf48-04fc574c7b44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:21.617916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:21.619125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:21.620317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 600/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:49:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:49:21.625069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9764d13-12f4-450e-886d-dc382a4e114f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:21.675019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:21.747766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9764d13-12f4-450e-886d-dc382a4e114f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:21.788514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:21.789121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:21.789394 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:21.789664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:21.790315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:21.790609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:21.792004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:21.800044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 609/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:49:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:49:21.805866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebb40d-81bf-42d1-bf48-04fc574c7b44 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:21.864289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:21.928865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebb40d-81bf-42d1-bf48-04fc574c7b44 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:21.969972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:21.970613 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:21.970916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:21.971192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:21.982274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:21.983978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:21.999143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:22.000966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:22.001837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:22.001837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:22.001837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:22.001837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 601/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:22.007418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:22.125350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:22.130380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb06ee8-bf5d-4a1a-8a91-10283dc77b50 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:22.177590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:22.180379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:22.180379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:22.180379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:22.180379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:22.180379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:22.181282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:22.194400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 610/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:49:22.200191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:22.318403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533464db-8bb6-496d-a657-f50532458cdc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:22.320786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:22.359437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:22.360211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:22.360597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:22.360956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:22.369436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:22.371063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:49:22.383913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:22.385333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:22.386559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:22.392849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 602/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:22.395548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:22.518905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd838c51-1864-40d1-a59d-b538dc9503b1 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:22.563643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:22.564255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:22.564567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:22.565015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:22.565861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:22.566200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:22.567901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:22.587666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 611/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Oct 10 11:49:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:22.594452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:22.708877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916735e-2372-4590-a2e7-bdd8bcc826ae None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:22.744471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1071e3b-b3ce-4ff5-8cb5-0bc7d022881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:22.745812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1071e3b-b3ce-4ff5-8cb5-0bc7d022881c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:22.745812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1071e3b-b3ce-4ff5-8cb5-0bc7d022881c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:22.745812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1071e3b-b3ce-4ff5-8cb5-0bc7d022881c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:22.776432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:22.816083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1071e3b-b3ce-4ff5-8cb5-0bc7d022881c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:22.817578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1071e3b-b3ce-4ff5-8cb5-0bc7d022881c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:22.926904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 603/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:49:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:49:22.932725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:22.953178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:23.041819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909eb73c-6cf7-4b02-ab26-394cf762af4d None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:23.082443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:23.083199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:23.083476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:23.083772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:23.096713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:23.098226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:23.116972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:23.118043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:23.127816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:23.127816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:23.127816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:23.127816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 612/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:23.128301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1071e3b-b3ce-4ff5-8cb5-0bc7d022881c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:23.202044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:23.252480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1071e3b-b3ce-4ff5-8cb5-0bc7d022881c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:23.291934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:23.292824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:23.293198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:23.293571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:23.294514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:23.295007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:23.297316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:23.311837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 604/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:23.319499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:23.416207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:23.456674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6b6b4-8eb8-4d80-93a0-b63f1c92aca8 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:23.500400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:23.501002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:23.501282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:23.501556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:23.509518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:23.511027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:49:23.524215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:23.527599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:23.528738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 613/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:49:23.533598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:23.642030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b3266-76fa-429f-8515-6fb14e18de3f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:23.665768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:23.678873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86e5d4-7c64-4a64-bb98-e463fa3e283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:23.679543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86e5d4-7c64-4a64-bb98-e463fa3e283e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:23.679829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86e5d4-7c64-4a64-bb98-e463fa3e283e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:23.680105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86e5d4-7c64-4a64-bb98-e463fa3e283e None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:23.680874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86e5d4-7c64-4a64-bb98-e463fa3e283e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:23.681127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86e5d4-7c64-4a64-bb98-e463fa3e283e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:23.682688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86e5d4-7c64-4a64-bb98-e463fa3e283e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:23.696693 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 605/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:49:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:23.703360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:23.818276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a62d54-a18d-48b1-9697-fcb9a69e2fc4 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:23.850735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:23.859703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf577a-1c97-45b8-84fd-d18f39dae864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:23.860377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf577a-1c97-45b8-84fd-d18f39dae864 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:23.860666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf577a-1c97-45b8-84fd-d18f39dae864 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:23.861008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf577a-1c97-45b8-84fd-d18f39dae864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:23.861829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf577a-1c97-45b8-84fd-d18f39dae864 None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=1e6c2f60d80647888998040638a5b993) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:23.862200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf577a-1c97-45b8-84fd-d18f39dae864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:23.863668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf577a-1c97-45b8-84fd-d18f39dae864 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:23.869676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 614/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:49:23 2022] GET /identity/v3/role_assignments?user.id=1e6c2f60d80647888998040638a5b993&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:26.074852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf7f68-169a-4286-84bd-a92a6597453f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:26.075891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf7f68-169a-4286-84bd-a92a6597453f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:26.075891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf7f68-169a-4286-84bd-a92a6597453f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:26.077135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 606/1220] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:26.082485 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec6a3-36c5-430c-82d5-ca06533bc787 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:26.082995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec6a3-36c5-430c-82d5-ca06533bc787 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:26.083096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec6a3-36c5-430c-82d5-ca06533bc787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:26.103775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-4b0ec6a3-36c5-430c-82d5-ca06533bc787 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:26.105659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4b0ec6a3-36c5-430c-82d5-ca06533bc787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:26.191342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 615/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:26.502242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf7f68-169a-4286-84bd-a92a6597453f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:26.616432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf7f68-169a-4286-84bd-a92a6597453f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:26.654850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f0652-b667-472f-8bf4-3600eb7227d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:26.655628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f0652-b667-472f-8bf4-3600eb7227d8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:26.655909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f0652-b667-472f-8bf4-3600eb7227d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:26.656179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f0652-b667-472f-8bf4-3600eb7227d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:26.667332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f0652-b667-472f-8bf4-3600eb7227d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-301f0652-b667-472f-8bf4-3600eb7227d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:26.672297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:26.673644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:26.674667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 607/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:26.679513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec6a3-36c5-430c-82d5-ca06533bc787 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:26.789751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec6a3-36c5-430c-82d5-ca06533bc787 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:26.813372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:26.824680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:26.825292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:26.825570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:26.825842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:26.826563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] RBAC: Adding query filter params (name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:26.826949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:26.828845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:26.837717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 616/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:49:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:26.843759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f0652-b667-472f-8bf4-3600eb7227d8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:26.889905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:26.949573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f0652-b667-472f-8bf4-3600eb7227d8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:26.991016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:26.991771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:26.992080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:26.992436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:27.003540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:27.004957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:27.019686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:27.025461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:27.026654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:27.026654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:27.026654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:27.026654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 608/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:27.026982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:27.138020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f83516-7ed6-4fdc-95d0-be0af1e6ab41 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:27.177376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:27.177973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:27.178341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:27.178525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:27.179196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:27.179504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:27.180867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:27.193297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 617/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:27.199075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:27.307981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:27.316322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fb5f4-f148-4828-80b4-657c271c48b2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:27.352566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:27.353339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:27.353712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:27.354035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:27.363212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:27.364839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:49:27.378105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:27.379350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:27.380725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 609/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:49:27.385494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:27.437603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:27.497956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3a856-e811-43c7-89e9-9e9697f7d704 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:27.534232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:27.534838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:27.535151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:27.535427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:27.536148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:27.536695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:27.538175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:27.557312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 618/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Oct 10 11:49:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:27.564299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:27.668878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3389b-f21b-4796-8dee-ddf37fa7862d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:27.706581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:27.707482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:27.707770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:27.708131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:27.719100 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:27.720530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:27.735275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:27.736221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:27.737425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:27.737425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:27.737425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:27.737425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 610/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:27.742212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:27.855843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e415a-700f-4241-91ab-acc0bcfbfad3 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:27.858294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:27.894762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:27.895677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:27.896020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:27.896451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:27.897301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:27.897624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:27.899813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:27.914145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 619/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:27.920017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:27.932464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:28.025520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22d2bd-d5ff-461c-9748-0a47ee7b57c1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:28.063114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:28.063745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:28.064026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:28.064297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:28.071046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:28.072562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:49:28.084779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:28.086359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:28.087308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 611/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:49:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:28.092411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:28.203441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f4ffe-041f-4f8a-a85e-04e004089721 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:28.243377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff5a29-65d3-4f38-aab0-8cc55f339b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:28.243982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5a29-65d3-4f38-aab0-8cc55f339b6b None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:28.244264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5a29-65d3-4f38-aab0-8cc55f339b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:28.244537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5a29-65d3-4f38-aab0-8cc55f339b6b None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:28.245224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff5a29-65d3-4f38-aab0-8cc55f339b6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:28.245531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff5a29-65d3-4f38-aab0-8cc55f339b6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:28.246979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff5a29-65d3-4f38-aab0-8cc55f339b6b None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:28.260198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 620/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:28.267093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:28.345590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:28.384406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224f6d-ee22-45b4-a1bc-e4ee9447bace None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:28.422009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24338-ad0b-48ab-a9b2-5621b49e6016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:28.422666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24338-ad0b-48ab-a9b2-5621b49e6016 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:28.422979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24338-ad0b-48ab-a9b2-5621b49e6016 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:28.423253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24338-ad0b-48ab-a9b2-5621b49e6016 None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/1e6c2f60d80647888998040638a5b993/roles/1b16ca28088b4fb4a1e9cc15fea47675` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:28.440558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24338-ad0b-48ab-a9b2-5621b49e6016 None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=1e6c2f60d80647888998040638a5b993, role_id=1b16ca28088b4fb4a1e9cc15fea47675)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:28.441995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24338-ad0b-48ab-a9b2-5621b49e6016 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:28.454763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 612/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:49:28 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/1e6c2f60d80647888998040638a5b993/roles/1b16ca28088b4fb4a1e9cc15fea47675 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:49:30.793380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d2fce-3a43-4063-a51e-8f8d7bf4e45c None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:30.793939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d2fce-3a43-4063-a51e-8f8d7bf4e45c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:30.793939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d2fce-3a43-4063-a51e-8f8d7bf4e45c None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:30.795206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 621/1233] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:30.800422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0385-0d87-4ab3-92d4-9af08316b7d6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:30.800822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0385-0d87-4ab3-92d4-9af08316b7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:30.800969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0385-0d87-4ab3-92d4-9af08316b7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:30.819673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e8ae0385-0d87-4ab3-92d4-9af08316b7d6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:30.820851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e8ae0385-0d87-4ab3-92d4-9af08316b7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:30.891343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 613/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:31.188952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d2fce-3a43-4063-a51e-8f8d7bf4e45c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:31.215627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:31.300740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d2fce-3a43-4063-a51e-8f8d7bf4e45c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:31.336719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110d0db-bd41-461e-bccb-bdcf077e897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:31.337362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d0db-bd41-461e-bccb-bdcf077e897a None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:31.337640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d0db-bd41-461e-bccb-bdcf077e897a None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:31.337905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d0db-bd41-461e-bccb-bdcf077e897a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:31.348976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110d0db-bd41-461e-bccb-bdcf077e897a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-8110d0db-bd41-461e-bccb-bdcf077e897a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:31.354345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:31.355531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:31.356510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 622/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Mon Oct 10 11:49:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:31.361587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae0385-0d87-4ab3-92d4-9af08316b7d6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:31.414014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:31.474911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae0385-0d87-4ab3-92d4-9af08316b7d6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:31.510042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:31.510641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:31.510946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:31.511219 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:31.511866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:31.512159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:31.513550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:31.526885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 614/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Mon Oct 10 11:49:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:31.528762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110d0db-bd41-461e-bccb-bdcf077e897a None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:31.621830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:31.640746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110d0db-bd41-461e-bccb-bdcf077e897a None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:31.678912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:31.679538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:31.679817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:31.680089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:31.691810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:31.693447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:31.711514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:31.712646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:31.713543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:31.713543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:31.713543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:31.713543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 623/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:31.719294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:31.828983 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192cafd-308f-4dcf-9737-78dc613b82c9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:31.838384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:31.867403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:31.868036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:31.868315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:31.868587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:31.869232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:31.869532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:31.870901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:31.884197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 615/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:31.890348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:31.998969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a9eaa-90ec-461d-8da7-62db69537c37 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:32.034456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:32.035119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:32.035396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:32.035672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:32.038469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:32.043674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:32.045120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:49:32.056201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:32.057514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:32.058644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 624/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:32.065058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:32.182523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bd645-274e-4ce5-b032-a527b75b3163 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:32.218437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:32.219096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:32.219359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:32.219629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:32.220338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:32.220646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:32.222068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:32.239949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 616/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Oct 10 11:49:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:32.245817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:32.348058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa84e1f-8914-47c0-8818-32100449c826 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:32.384695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbcdc1-57d8-48b8-8714-6f59a9a42ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:32.385358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbcdc1-57d8-48b8-8714-6f59a9a42ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:32.385687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbcdc1-57d8-48b8-8714-6f59a9a42ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:32.385986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbcdc1-57d8-48b8-8714-6f59a9a42ae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:32.458843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:32.465959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbcdc1-57d8-48b8-8714-6f59a9a42ae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:32.467602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbcdc1-57d8-48b8-8714-6f59a9a42ae2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:32.577938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 625/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:49:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:49:32.584357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:32.613877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:32.693930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae5cd1-035b-400c-bf23-42711afc8e4d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:32.732706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:32.733275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:32.734088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:32.734361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:32.745103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:32.746667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:32.767258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:32.768447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:32.769359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:32.769359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:32.769359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:32.769359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 617/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:32.775126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbcdc1-57d8-48b8-8714-6f59a9a42ae2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:32.875037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:32.881455 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbcdc1-57d8-48b8-8714-6f59a9a42ae2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:32.917886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:32.918590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:32.918760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:32.919058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:32.919698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:32.920001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:32.921392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:32.932806 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 626/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:32.938982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:33.024372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:33.049719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbb32d-ae3f-4e67-b98f-d87d3cc827d4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:33.084439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:33.085120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:33.085442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:33.085749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:33.097373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:33.099076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:49:33.113297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:33.114384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:33.120170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 618/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:49:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:33.120763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:33.243377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea66e21-f7c5-4a6b-a0f6-c6d4c775e90c None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:33.278814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4f4c7-b025-477b-84be-5a74fc2a6a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:33.279409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f4c7-b025-477b-84be-5a74fc2a6a29 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:33.279729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f4c7-b025-477b-84be-5a74fc2a6a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:33.280069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f4c7-b025-477b-84be-5a74fc2a6a29 None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:33.281456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4f4c7-b025-477b-84be-5a74fc2a6a29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:33.281773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4f4c7-b025-477b-84be-5a74fc2a6a29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:33.283481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4f4c7-b025-477b-84be-5a74fc2a6a29 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:33.297820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 627/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:49:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:33.304050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:33.428039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894813-aa25-4cc6-ad47-dd7e2b110f9c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:33.466089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ec0c9-2ee8-4ad7-958c-a53984fc421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:33.466691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ec0c9-2ee8-4ad7-958c-a53984fc421a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:33.467004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ec0c9-2ee8-4ad7-958c-a53984fc421a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:33.467277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ec0c9-2ee8-4ad7-958c-a53984fc421a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:33.468499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ec0c9-2ee8-4ad7-958c-a53984fc421a None admin] RBAC: Adding query filter params (role.id=1b16ca28088b4fb4a1e9cc15fea47675, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=1e6c2f60d80647888998040638a5b993) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:33.468801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ec0c9-2ee8-4ad7-958c-a53984fc421a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:33.470616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ec0c9-2ee8-4ad7-958c-a53984fc421a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:33.477913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 619/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:49:33 2022] GET /identity/v3/role_assignments?user.id=1e6c2f60d80647888998040638a5b993&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=1b16ca28088b4fb4a1e9cc15fea47675 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:49:35.975786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63a467-a759-42aa-a27f-17679f7f1934 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:35.976371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63a467-a759-42aa-a27f-17679f7f1934 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:35.976371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63a467-a759-42aa-a27f-17679f7f1934 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:35.977708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 628/1247] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:35.983057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a122e-6a62-4db7-b336-50b4ebda12c1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:35.983975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a122e-6a62-4db7-b336-50b4ebda12c1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:35.984166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a122e-6a62-4db7-b336-50b4ebda12c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:35.992058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:36.005447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e88a122e-6a62-4db7-b336-50b4ebda12c1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:36.006726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e88a122e-6a62-4db7-b336-50b4ebda12c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:36.080834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 620/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:36.382466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63a467-a759-42aa-a27f-17679f7f1934 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:36.402627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:36.499466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63a467-a759-42aa-a27f-17679f7f1934 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:36.539998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8338a-9a95-48fe-8f12-5f202f4edf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:36.540912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8338a-9a95-48fe-8f12-5f202f4edf70 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:36.541190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8338a-9a95-48fe-8f12-5f202f4edf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:36.541466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8338a-9a95-48fe-8f12-5f202f4edf70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:36.552991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8338a-9a95-48fe-8f12-5f202f4edf70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-96e8338a-9a95-48fe-8f12-5f202f4edf70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:36.557754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:36.559537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:36.560767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 629/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:49:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:36.566344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a122e-6a62-4db7-b336-50b4ebda12c1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:36.681450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a122e-6a62-4db7-b336-50b4ebda12c1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:36.714200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:36.720562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:36.721175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:36.721452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:36.721716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:36.722375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:36.722671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:36.724069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:36.732841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 621/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:49:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:36.738821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8338a-9a95-48fe-8f12-5f202f4edf70 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:36.821952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:36.846772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8338a-9a95-48fe-8f12-5f202f4edf70 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:36.881275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:36.881913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:36.882188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:36.882457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:36.893324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:36.894792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:36.912115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:36.913268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:36.914193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:36.914193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:36.914193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:36.914193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 630/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:36.919221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:37.024624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0a761-597d-4ccb-8019-cae5e25592ea None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:37.057689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:37.058298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:37.058577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:37.059197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:37.059838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:37.060140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:37.062011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:37.073524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 622/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:49:37.079986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:37.192058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b33aba-142e-496a-af6a-dfb83f5c7dc6 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:37.218148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:37.228414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:37.228414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:37.228414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:37.228892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:37.237002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:37.238426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:49:37.250253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:37.251387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:37.252524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 631/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:49:37.257444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:37.314282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:37.357513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3933-a2c7-4ce1-9a64-82ba8b010505 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:37.390071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:37.390663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:37.390969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:37.391244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:37.391970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:37.392274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:37.393675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:37.410665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 623/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Oct 10 11:49:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:49:37.416436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:37.512081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a726e3-fea6-42f4-9039-800b24526778 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:37.544829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58130-1b60-42aa-a9e0-37461c258b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:37.545424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58130-1b60-42aa-a9e0-37461c258b45 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:37.545723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58130-1b60-42aa-a9e0-37461c258b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:37.546004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58130-1b60-42aa-a9e0-37461c258b45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:37.599141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:37.611765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd58130-1b60-42aa-a9e0-37461c258b45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:37.613184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd58130-1b60-42aa-a9e0-37461c258b45 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:37.708247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 632/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:49:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:49:37.714014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:37.808619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247b932-eb91-44d6-89ec-7f93d745e8fe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:37.821850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:37.843067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:37.843683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:37.843957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:37.844225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:37.853856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:37.855250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:37.869348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:37.870429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:37.871518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:37.871518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:37.871518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:37.871518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 624/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:37.876480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58130-1b60-42aa-a9e0-37461c258b45 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:37.961082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:37.973276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58130-1b60-42aa-a9e0-37461c258b45 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:38.006456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:38.007073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:38.007358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:38.007626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:38.008253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:38.008550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:38.009903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:38.021126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 633/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:38.026657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:38.125527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfdcc-0841-4f13-9bdc-abb0b07732bf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:38.158488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:38.159123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:38.159402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:38.159669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:38.165992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:38.167456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:49:38.178981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:38.179877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:38.180956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 625/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:38.186154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:38.282513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c372704-bf5c-4b74-a1d4-683634bce7e5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:38.314414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b79b4-2eea-4a32-ab82-879f628bac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:38.315027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b79b4-2eea-4a32-ab82-879f628bac9d None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:38.315388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b79b4-2eea-4a32-ab82-879f628bac9d None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:38.315653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b79b4-2eea-4a32-ab82-879f628bac9d None admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:38.316327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b79b4-2eea-4a32-ab82-879f628bac9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:38.316627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b79b4-2eea-4a32-ab82-879f628bac9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:38.318023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b79b4-2eea-4a32-ab82-879f628bac9d None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:38.329889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 634/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:38.335700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:38.360724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:38.432336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105a628-2a0c-4b5e-a445-d708b4f29819 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:38.465094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78d54f-9206-4b99-830e-26da301d8456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:38.465680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78d54f-9206-4b99-830e-26da301d8456 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:38.465958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78d54f-9206-4b99-830e-26da301d8456 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:38.466226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78d54f-9206-4b99-830e-26da301d8456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:38.466960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78d54f-9206-4b99-830e-26da301d8456 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=1e6c2f60d80647888998040638a5b993) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:38.467264 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78d54f-9206-4b99-830e-26da301d8456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:38.469194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78d54f-9206-4b99-830e-26da301d8456 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:38.474335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 626/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:49:38 2022] GET /identity/v3/role_assignments?user.id=1e6c2f60d80647888998040638a5b993&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:40.571383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a555b8-3e04-4581-abe9-9ba26b234255 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:40.571899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a555b8-3e04-4581-abe9-9ba26b234255 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:40.571899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a555b8-3e04-4581-abe9-9ba26b234255 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:40.573251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 635/1261] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:40.578864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24331862-fc6d-4cc4-ae98-386265488fd5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:40.579289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24331862-fc6d-4cc4-ae98-386265488fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:40.579451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24331862-fc6d-4cc4-ae98-386265488fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:40.597593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-24331862-fc6d-4cc4-ae98-386265488fd5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:40.598768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-24331862-fc6d-4cc4-ae98-386265488fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:40.663047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 627/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:40.928748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a555b8-3e04-4581-abe9-9ba26b234255 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:40.939693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:41.026577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a555b8-3e04-4581-abe9-9ba26b234255 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:41.058934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90e6ee-17b8-4af3-96fc-9a2ecb1da880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:41.059548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90e6ee-17b8-4af3-96fc-9a2ecb1da880 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:41.059825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90e6ee-17b8-4af3-96fc-9a2ecb1da880 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:41.060096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90e6ee-17b8-4af3-96fc-9a2ecb1da880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:41.069629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90e6ee-17b8-4af3-96fc-9a2ecb1da880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-3e90e6ee-17b8-4af3-96fc-9a2ecb1da880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:41.074243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:41.075351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:41.076510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 636/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:49:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:41.081700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24331862-fc6d-4cc4-ae98-386265488fd5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:41.101232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:41.189666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24331862-fc6d-4cc4-ae98-386265488fd5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:41.221486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:41.222090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:41.222365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:41.222631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:41.223310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:41.223609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:41.224949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:41.232575 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 628/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:41.238950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90e6ee-17b8-4af3-96fc-9a2ecb1da880 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:41.302543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:41.334611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90e6ee-17b8-4af3-96fc-9a2ecb1da880 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:41.367972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:41.368580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:41.368855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:41.369122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:41.378527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:41.379972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:41.393818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:41.394940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:41.396063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:41.396063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:41.396063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:41.396063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 637/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:41.400784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:41.480223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:41.499565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8b788-bf0e-4ca2-9cca-946b704ab751 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:41.532036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:41.532623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:41.532900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:41.533170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:41.533801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:41.534099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:41.535457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:41.547190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 629/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:41.552984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:41.649455 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e9b69-6742-4b31-aaab-8ccea556b892 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:41.668307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:41.682611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:41.683238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:41.683516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:41.684033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:41.691094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:41.692508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:49:41.703133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:41.704178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:41.705524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 638/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:41.709936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:41.805184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e0f69-1840-4eca-ab2a-c0d50bf53533 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:41.837708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:41.838290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:41.838567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:41.838834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:41.839576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:41.839876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:41.841266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:41.849586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:41.859432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 630/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Oct 10 11:49:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:41.865015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:41.962034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fb868-49e7-4d6e-ac24-88f09b844d7e None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:41.993752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:41.994355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:41.994627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:41.994924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:42.004913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:42.006324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:42.020167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:42.021245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:42.022152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:42.022152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:42.022152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:42.022152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 639/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:42.026759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:42.131144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec6dcc-3b62-4fda-8470-99bae7ab5347 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:42.163997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:42.164593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:42.164869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:42.165144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:42.165775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:42.166073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:42.167451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:42.178256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 631/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:42.184154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:42.215242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:42.280422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb20491-c522-4914-b424-1fc7c2dd04e5 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:42.312162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:42.312760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:42.313034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:42.313299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:42.319491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:42.320896 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:49:42.332109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:42.333120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:42.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 640/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:42.338731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:42.394859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:42.435211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ddd95-d7dc-403a-be18-3b6980c87a4c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:42.468271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f32a35-b5d3-4862-a0bd-5b787b4edfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:42.468865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f32a35-b5d3-4862-a0bd-5b787b4edfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:42.469145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f32a35-b5d3-4862-a0bd-5b787b4edfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:42.469416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f32a35-b5d3-4862-a0bd-5b787b4edfe6 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:42.470097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f32a35-b5d3-4862-a0bd-5b787b4edfe6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:42.470394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f32a35-b5d3-4862-a0bd-5b787b4edfe6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:42.471823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f32a35-b5d3-4862-a0bd-5b787b4edfe6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:42.483797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 632/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:42.489488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:42.576602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:42.585898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b04b01-6a2d-4d62-a186-2e536e55cbbf None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:42.617944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d497734-7124-4666-ba40-4e66c0eccd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:42.618579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d497734-7124-4666-ba40-4e66c0eccd48 None admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:42.618853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d497734-7124-4666-ba40-4e66c0eccd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:42.619157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d497734-7124-4666-ba40-4e66c0eccd48 None admin] PATH_INFO: `/v3/projects/fad1c962c6024e5cbb001d3256203928/users/1e6c2f60d80647888998040638a5b993/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:42.620975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d497734-7124-4666-ba40-4e66c0eccd48 None admin] RBAC: Authorizing `identity:create_grant(project_id=fad1c962c6024e5cbb001d3256203928, user_id=1e6c2f60d80647888998040638a5b993, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:42.622383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d497734-7124-4666-ba40-4e66c0eccd48 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:42.634459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 641/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Mon Oct 10 11:49:42 2022] PUT /identity/v3/projects/fad1c962c6024e5cbb001d3256203928/users/1e6c2f60d80647888998040638a5b993/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:49:44.754253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190203-6987-440d-b2cd-1b5e8ecbd545 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:44.754780 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190203-6987-440d-b2cd-1b5e8ecbd545 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:44.754780 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190203-6987-440d-b2cd-1b5e8ecbd545 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:44.756061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 633/1274] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:44.761114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b61b-393d-4209-8d50-e6ae03d3cf4b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:44.761526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b61b-393d-4209-8d50-e6ae03d3cf4b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:44.761667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b61b-393d-4209-8d50-e6ae03d3cf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:44.779386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-7a88b61b-393d-4209-8d50-e6ae03d3cf4b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:44.780570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7a88b61b-393d-4209-8d50-e6ae03d3cf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:44.849239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 642/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:45.119018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c190203-6987-440d-b2cd-1b5e8ecbd545 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:45.223105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c190203-6987-440d-b2cd-1b5e8ecbd545 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:45.239245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:45.255718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd370a-fe67-42e0-88e6-593d76bf4724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:45.256332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd370a-fe67-42e0-88e6-593d76bf4724 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:45.256609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd370a-fe67-42e0-88e6-593d76bf4724 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:45.256878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd370a-fe67-42e0-88e6-593d76bf4724 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:45.267023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd370a-fe67-42e0-88e6-593d76bf4724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7bbd370a-fe67-42e0-88e6-593d76bf4724 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:45.271637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:45.272745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:45.273789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 634/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Mon Oct 10 11:49:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:45.278767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88b61b-393d-4209-8d50-e6ae03d3cf4b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:45.355726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:45.374677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88b61b-393d-4209-8d50-e6ae03d3cf4b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:45.407493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:45.408090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:45.408369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:45.408639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:45.409289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:45.409590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:45.410985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:45.418984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 643/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Mon Oct 10 11:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:45.424886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd370a-fe67-42e0-88e6-593d76bf4724 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:45.523207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd370a-fe67-42e0-88e6-593d76bf4724 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:45.555884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:45.556491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:45.556773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:45.557046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:45.566916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:45.568357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:45.582968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:45.584103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:45.585323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:45.585323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:45.585323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:45.585323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 635/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:45.589552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:45.685702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9b51f-ef67-4d1b-84a6-147f2240eebe None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:45.718271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:45.718860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:45.719164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:45.719436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:45.720062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:45.720362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:45.721718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:45.732861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 644/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:45.738396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:45.769299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:45.837102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4a030-285d-4061-a935-d1800dcf3df5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:45.869629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:45.870238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:45.870518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:45.870787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:45.878378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:45.879823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:49:45.891238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:45.892331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:49:45.893602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:45.895043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 636/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:45.897832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:45.901483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:45.993878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639654d-715d-4304-b3ba-3f9469e775d0 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:46.026662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:46.027280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:46.027560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:46.027832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:46.028543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:46.028846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:46.030768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:46.047391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 645/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Mon Oct 10 11:49:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:46.053103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:46.142802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:46.152295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2070e68-2b16-4305-87b6-9b2bc7114d15 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:46.184827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7a719-1143-437b-908f-c1076df4c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:46.185412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7a719-1143-437b-908f-c1076df4c85c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:46.185686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7a719-1143-437b-908f-c1076df4c85c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:46.185953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7a719-1143-437b-908f-c1076df4c85c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:46.249392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7a719-1143-437b-908f-c1076df4c85c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:46.250763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7a719-1143-437b-908f-c1076df4c85c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:46.334643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:46.345313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 637/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Mon Oct 10 11:49:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:49:46.351092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:46.410553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:46.448445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c28177-4810-4dbb-8f9f-5833c55a4ea8 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:46.482357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:46.483038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:46.483323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:46.483593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:46.493196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:46.494621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:46.508697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:46.509888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:49:46.511220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:46.511220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:49:46.511220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:46.511220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 646/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:46.515890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7a719-1143-437b-908f-c1076df4c85c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:46.610166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7a719-1143-437b-908f-c1076df4c85c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:46.643033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:46.643624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:46.643902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:46.644171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:46.644798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:46.645096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:46.646433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:46.657106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 638/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Mon Oct 10 11:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:46.662580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:46.757960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4fda9-c0f5-4c7c-a877-8db2c9a5caa7 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:46.771408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:46.790338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:46.790974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:46.791247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:46.791510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:46.797823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:46.799262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:49:46.810110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:46.811674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: ERROR keystone.server.flask.application Oct 10 11:49:46.812942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 647/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Mon Oct 10 11:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:46.817418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:46.865193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:46.913972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175f963-a8c9-40cb-adec-6ab8fe9d6fb8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:46.946197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b7890-9587-47f6-94c5-973494395886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:46.946787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7890-9587-47f6-94c5-973494395886 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:46.947096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7890-9587-47f6-94c5-973494395886 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:46.947368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7890-9587-47f6-94c5-973494395886 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:46.948055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b7890-9587-47f6-94c5-973494395886 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:46.948354 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b7890-9587-47f6-94c5-973494395886 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:46.949760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b7890-9587-47f6-94c5-973494395886 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:46.962292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 639/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Mon Oct 10 11:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:46.968236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:47.064143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65daa6-6443-4fc5-a3f4-7a90e39a76f2 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:47.097813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126adf2c-1153-42a3-a5f4-25d7d2d6dd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:47.098395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126adf2c-1153-42a3-a5f4-25d7d2d6dd27 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:47.098670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126adf2c-1153-42a3-a5f4-25d7d2d6dd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:47.098961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126adf2c-1153-42a3-a5f4-25d7d2d6dd27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:47.099667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126adf2c-1153-42a3-a5f4-25d7d2d6dd27 None admin] RBAC: Adding query filter params (role.id=2852c997f1e640a7afa94b5f113b12d6, scope.project.id=fad1c962c6024e5cbb001d3256203928, user.id=1e6c2f60d80647888998040638a5b993) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:47.099968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126adf2c-1153-42a3-a5f4-25d7d2d6dd27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:47.101369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126adf2c-1153-42a3-a5f4-25d7d2d6dd27 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:47.106810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 648/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Mon Oct 10 11:49:46 2022] GET /identity/v3/role_assignments?user.id=1e6c2f60d80647888998040638a5b993&scope.project.id=fad1c962c6024e5cbb001d3256203928&role.id=2852c997f1e640a7afa94b5f113b12d6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:49.239470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b392f-5a44-43db-9a95-e0562ee1b1a4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:49.239992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b392f-5a44-43db-9a95-e0562ee1b1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:49.239992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b392f-5a44-43db-9a95-e0562ee1b1a4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:49.241354 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 640/1288] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:49.246359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a218d-19d3-42c3-b4e4-b78eb9ca8234 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:49.246754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a218d-19d3-42c3-b4e4-b78eb9ca8234 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:49.246914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a218d-19d3-42c3-b4e4-b78eb9ca8234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:49.265022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f76a218d-19d3-42c3-b4e4-b78eb9ca8234 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:49.266180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f76a218d-19d3-42c3-b4e4-b78eb9ca8234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:49.293900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:49.330209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 649/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:49.596417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b392f-5a44-43db-9a95-e0562ee1b1a4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:49.692028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b392f-5a44-43db-9a95-e0562ee1b1a4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:49.702037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:49.726363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:49.727003 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:49.727282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:49.727547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:49.728117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:49.729448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:49.738319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:49.739435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:49.740285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 641/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:49:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:49.745437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a218d-19d3-42c3-b4e4-b78eb9ca8234 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:49.839734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a218d-19d3-42c3-b4e4-b78eb9ca8234 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:49.872081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11547880-9f8d-4b94-915f-bac23a63e327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:49.872606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11547880-9f8d-4b94-915f-bac23a63e327 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:49.872882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11547880-9f8d-4b94-915f-bac23a63e327 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:49.873151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11547880-9f8d-4b94-915f-bac23a63e327 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:49.873785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11547880-9f8d-4b94-915f-bac23a63e327 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:49.874083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11547880-9f8d-4b94-915f-bac23a63e327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:49.875454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11547880-9f8d-4b94-915f-bac23a63e327 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:49.880007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 650/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Oct 10 11:49:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:49.885469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:49.981345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4d4f1-b930-4ad1-ab38-36d80098bd3a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:50.013673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b601505-58f5-4fd7-9e3c-a5400daf293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:50.014767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b601505-58f5-4fd7-9e3c-a5400daf293e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:50.015071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b601505-58f5-4fd7-9e3c-a5400daf293e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:50.015342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b601505-58f5-4fd7-9e3c-a5400daf293e None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:50.015953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b601505-58f5-4fd7-9e3c-a5400daf293e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:50.016251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b601505-58f5-4fd7-9e3c-a5400daf293e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:50.017590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b601505-58f5-4fd7-9e3c-a5400daf293e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:50.021833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 642/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Oct 10 11:49:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:52.160233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360d584-3f3c-4736-84c4-5d8a592500f5 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:52.160785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360d584-3f3c-4736-84c4-5d8a592500f5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:52.160785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360d584-3f3c-4736-84c4-5d8a592500f5 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:52.162115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 651/1293] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:52.167371 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040842bb-dd72-4dd2-8172-dae4ce20ea74 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:52.167786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040842bb-dd72-4dd2-8172-dae4ce20ea74 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:52.167922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040842bb-dd72-4dd2-8172-dae4ce20ea74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:52.186424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-040842bb-dd72-4dd2-8172-dae4ce20ea74 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:52.187651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-040842bb-dd72-4dd2-8172-dae4ce20ea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:52.216115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:52.252598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 643/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:52.515559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360d584-3f3c-4736-84c4-5d8a592500f5 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:52.546117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:52.615516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360d584-3f3c-4736-84c4-5d8a592500f5 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:52.648907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34bd8a-7c28-4907-ba19-0ee6c4c20b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:52.649499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34bd8a-7c28-4907-ba19-0ee6c4c20b86 None admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:52.649780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34bd8a-7c28-4907-ba19-0ee6c4c20b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:52.650106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34bd8a-7c28-4907-ba19-0ee6c4c20b86 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:52.650760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34bd8a-7c28-4907-ba19-0ee6c4c20b86 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:52.652145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34bd8a-7c28-4907-ba19-0ee6c4c20b86 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:52.666463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 652/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Mon Oct 10 11:49:52 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:49:54.817714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96660ddd-b606-4519-9616-bff605e7b4e4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:54.818209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96660ddd-b606-4519-9616-bff605e7b4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:54.818209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96660ddd-b606-4519-9616-bff605e7b4e4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:54.819551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 644/1296] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:54.824816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb889-24e9-4517-911b-fd7e1acf3ed0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:54.825248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb889-24e9-4517-911b-fd7e1acf3ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:54.825342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb889-24e9-4517-911b-fd7e1acf3ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:54.843886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3c0fb889-24e9-4517-911b-fd7e1acf3ed0 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:54.845040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3c0fb889-24e9-4517-911b-fd7e1acf3ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:54.914446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 653/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:55.180116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96660ddd-b606-4519-9616-bff605e7b4e4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:55.278257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96660ddd-b606-4519-9616-bff605e7b4e4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:55.311605 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:55.312212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:55.312489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:55.312757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:55.313329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:55.314673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:55.323215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:55.324353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:55.325519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 645/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:49:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:49:55.330091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fb889-24e9-4517-911b-fd7e1acf3ed0 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:55.347630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:55.428210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fb889-24e9-4517-911b-fd7e1acf3ed0 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:55.461237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:55.461776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:55.462058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:55.462333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:55.462987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:55.463293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:55.464646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:55.469587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 654/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Oct 10 11:49:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:55.475541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:55.506988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:55.573396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0980f9-6a1b-403d-89da-1286037d5162 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:55.605320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ce6e7-edf9-4a41-8393-7c08176ab192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:55.605903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce6e7-edf9-4a41-8393-7c08176ab192 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:55.606183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce6e7-edf9-4a41-8393-7c08176ab192 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:55.606455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce6e7-edf9-4a41-8393-7c08176ab192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:55.607169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ce6e7-edf9-4a41-8393-7c08176ab192 None admin] RBAC: Adding query filter params (interface=public, service_id=70bf34c416ea4f4088c28df9f4f3d268, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:55.607473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ce6e7-edf9-4a41-8393-7c08176ab192 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:55.609354 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ce6e7-edf9-4a41-8393-7c08176ab192 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:55.614116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 646/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Mon Oct 10 11:49:55 2022] GET /identity/v3/endpoints?service_id=70bf34c416ea4f4088c28df9f4f3d268&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:55.619836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:55.698286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:55.716821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc4b5f-e33a-4a74-b389-087fd64f9207 None admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:55.748754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4c9ca-90d4-4949-a0c3-8b2a15b563b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:55.749337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4c9ca-90d4-4949-a0c3-8b2a15b563b9 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:55.749614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4c9ca-90d4-4949-a0c3-8b2a15b563b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:55.749882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4c9ca-90d4-4949-a0c3-8b2a15b563b9 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:55.750469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4c9ca-90d4-4949-a0c3-8b2a15b563b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:55.751853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4c9ca-90d4-4949-a0c3-8b2a15b563b9 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:55.758363 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 655/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Mon Oct 10 11:49:55 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:49:57.864718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be264a12-d116-4ce6-90b5-7ded7a45f645 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:57.865267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be264a12-d116-4ce6-90b5-7ded7a45f645 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:57.865267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be264a12-d116-4ce6-90b5-7ded7a45f645 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:57.866590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 647/1302] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:57.871920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fec37-1a65-44f0-92eb-b33f6d306845 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:57.872335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fec37-1a65-44f0-92eb-b33f6d306845 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:57.872448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fec37-1a65-44f0-92eb-b33f6d306845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:57.891077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-742fec37-1a65-44f0-92eb-b33f6d306845 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:57.892238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-742fec37-1a65-44f0-92eb-b33f6d306845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:57.955200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 656/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:58.222439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be264a12-d116-4ce6-90b5-7ded7a45f645 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:58.314835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:58.321507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be264a12-d116-4ce6-90b5-7ded7a45f645 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:58.353982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:58.354588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:58.354865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:58.355170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:58.355745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:58.357093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:58.365491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:58.366724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Oct 10 11:49:58.367682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 648/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Mon Oct 10 11:49:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:58.372382 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fec37-1a65-44f0-92eb-b33f6d306845 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:58.444819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:58.467781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fec37-1a65-44f0-92eb-b33f6d306845 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:58.499349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76453f-2fd9-491d-a55e-2132636d3005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:58.499890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76453f-2fd9-491d-a55e-2132636d3005 None admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:58.500166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76453f-2fd9-491d-a55e-2132636d3005 None admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:58.500434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76453f-2fd9-491d-a55e-2132636d3005 None admin] PATH_INFO: `/v3/services` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:58.501047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76453f-2fd9-491d-a55e-2132636d3005 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:58.501342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76453f-2fd9-491d-a55e-2132636d3005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:58.502687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76453f-2fd9-491d-a55e-2132636d3005 None admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:58.507592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 657/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Mon Oct 10 11:49:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:58.513326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:58.609981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa34fc-647f-4490-a857-ccdc585bf9a6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:58.642569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb27701-f074-46eb-8878-62cf1042df49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:58.643170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27701-f074-46eb-8878-62cf1042df49 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:58.643445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27701-f074-46eb-8878-62cf1042df49 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:58.643711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27701-f074-46eb-8878-62cf1042df49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:58.644352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb27701-f074-46eb-8878-62cf1042df49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:58.645852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb27701-f074-46eb-8878-62cf1042df49 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:58.664480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 649/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:49:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:50:08.504795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=30 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:08.660709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:01.821573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4360fd-b4cf-4d78-b6cf-c1df7022d041 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:01.822063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4360fd-b4cf-4d78-b6cf-c1df7022d041 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:01.822158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4360fd-b4cf-4d78-b6cf-c1df7022d041 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:01.829733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5338e-7092-4ddc-acb6-6dd50026304b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:01.830168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5338e-7092-4ddc-acb6-6dd50026304b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:01.830276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5338e-7092-4ddc-acb6-6dd50026304b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:01.834939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 658/1307] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:01 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:01.924621 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6ac5338e-7092-4ddc-acb6-6dd50026304b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:01.925913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6ac5338e-7092-4ddc-acb6-6dd50026304b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:02.038935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 650/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:02.335185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14fd33-ba9b-432e-8455-08cf370e41ad None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.335516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14fd33-ba9b-432e-8455-08cf370e41ad None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.335750 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14fd33-ba9b-432e-8455-08cf370e41ad None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.337123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 659/1309] 10.222.0.52 () {54 vars in 964 bytes} [Mon Oct 10 11:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:02.343473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da5e48-6dca-422a-afc8-49875c68e6e5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.343858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da5e48-6dca-422a-afc8-49875c68e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.344208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da5e48-6dca-422a-afc8-49875c68e6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.436963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-13da5e48-6dca-422a-afc8-49875c68e6e5 None None] MFA Rules not processed for user `48ad70aa047d443b97e13447397a8a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:02.438172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-13da5e48-6dca-422a-afc8-49875c68e6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:02.460457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:02.526501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 651/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Mon Oct 10 11:51:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:02.536110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14fd33-ba9b-432e-8455-08cf370e41ad None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:02.538052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1f14fd33-ba9b-432e-8455-08cf370e41ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:02.656483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14fd33-ba9b-432e-8455-08cf370e41ad None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:02.735322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f8f75-0e8a-4375-9eb6-1cac68014b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:02.735928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f8f75-0e8a-4375-9eb6-1cac68014b59 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.736207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f8f75-0e8a-4375-9eb6-1cac68014b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.736478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f8f75-0e8a-4375-9eb6-1cac68014b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.738074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-250f8f75-0e8a-4375-9eb6-1cac68014b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:02.794581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:02.852633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f8f75-0e8a-4375-9eb6-1cac68014b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:02.854066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f8f75-0e8a-4375-9eb6-1cac68014b59 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:02.872989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 660/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:51:05.349895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61747c29-51df-49dd-86bb-dd6a3e0943ea None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:05.350346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61747c29-51df-49dd-86bb-dd6a3e0943ea None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:05.350588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61747c29-51df-49dd-86bb-dd6a3e0943ea None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:05.352016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 652/1312] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:05.357603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14429216-7322-4d75-a4cd-ce75055f6e6f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:05.357979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14429216-7322-4d75-a4cd-ce75055f6e6f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:05.358167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14429216-7322-4d75-a4cd-ce75055f6e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:05.433113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-14429216-7322-4d75-a4cd-ce75055f6e6f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:05.434328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-14429216-7322-4d75-a4cd-ce75055f6e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:05.529359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 661/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:05.826347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be6818-ef17-4666-a22f-63baeaf7a17d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:05.826634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be6818-ef17-4666-a22f-63baeaf7a17d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:05.826934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be6818-ef17-4666-a22f-63baeaf7a17d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:05.828357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 653/1314] 10.222.0.52 () {54 vars in 964 bytes} [Mon Oct 10 11:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:05.835034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f792a-ca5f-4394-a4ca-7c695b8c908c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:05.835328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f792a-ca5f-4394-a4ca-7c695b8c908c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:05.835602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f792a-ca5f-4394-a4ca-7c695b8c908c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:05.850146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:05.919935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-589f792a-ca5f-4394-a4ca-7c695b8c908c None None] MFA Rules not processed for user `48ad70aa047d443b97e13447397a8a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:05.921163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-589f792a-ca5f-4394-a4ca-7c695b8c908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:06.015280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 662/1315] 10.222.0.52 () {58 vars in 1061 bytes} [Mon Oct 10 11:51:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:06.031502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be6818-ef17-4666-a22f-63baeaf7a17d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:06.033280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-58be6818-ef17-4666-a22f-63baeaf7a17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:06.152188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be6818-ef17-4666-a22f-63baeaf7a17d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:06.223272 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:06.225645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1a724-0d9b-41c3-80a8-eda8d28aac53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:06.226475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1a724-0d9b-41c3-80a8-eda8d28aac53 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:06.226966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1a724-0d9b-41c3-80a8-eda8d28aac53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:06.227788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1a724-0d9b-41c3-80a8-eda8d28aac53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:06.229425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cab1a724-0d9b-41c3-80a8-eda8d28aac53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:06.336393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1a724-0d9b-41c3-80a8-eda8d28aac53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:06.337822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1a724-0d9b-41c3-80a8-eda8d28aac53 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:06.357418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 654/1316] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:09.013628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38609a-e61d-4d5b-9ad0-ec95f801ee3e None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:09.014162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38609a-e61d-4d5b-9ad0-ec95f801ee3e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:09.014162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38609a-e61d-4d5b-9ad0-ec95f801ee3e None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:09.015575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 663/1317] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:09.020869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd1334-39f3-4229-8e03-92552756cf5a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:09.021268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd1334-39f3-4229-8e03-92552756cf5a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:09.021414 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd1334-39f3-4229-8e03-92552756cf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:09.087080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-acfd1334-39f3-4229-8e03-92552756cf5a None None] MFA Rules not processed for user `4dbfa2988ba9444197b16560d6e65096`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:09.088272 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-acfd1334-39f3-4229-8e03-92552756cf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:09.140636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:09.184526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 655/1318] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:09 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:09.469945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38609a-e61d-4d5b-9ad0-ec95f801ee3e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:09.471500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3f38609a-e61d-4d5b-9ad0-ec95f801ee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:09.571875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38609a-e61d-4d5b-9ad0-ec95f801ee3e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:09.575137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:09.644060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1310a70-4022-49fa-90d7-25aa3f06e375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:09.644662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1310a70-4022-49fa-90d7-25aa3f06e375 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:09.644938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1310a70-4022-49fa-90d7-25aa3f06e375 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:09.645204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1310a70-4022-49fa-90d7-25aa3f06e375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:09.646819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a1310a70-4022-49fa-90d7-25aa3f06e375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:09.751352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1310a70-4022-49fa-90d7-25aa3f06e375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:09.752767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1310a70-4022-49fa-90d7-25aa3f06e375 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:09.771838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 664/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:12.428311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1863b3-1f9b-430d-91c0-7dbed062366a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:12.428769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1863b3-1f9b-430d-91c0-7dbed062366a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:12.428865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1863b3-1f9b-430d-91c0-7dbed062366a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:12.430181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 656/1320] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:12.436099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798c51d-700a-448e-9eb2-410b90d4d9aa None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:12.436544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798c51d-700a-448e-9eb2-410b90d4d9aa None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:12.436640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798c51d-700a-448e-9eb2-410b90d4d9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:12.476553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:12.501842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-2798c51d-700a-448e-9eb2-410b90d4d9aa None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:12.503071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2798c51d-700a-448e-9eb2-410b90d4d9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:12.595349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 665/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:12.876135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1863b3-1f9b-430d-91c0-7dbed062366a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:12.877688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5a1863b3-1f9b-430d-91c0-7dbed062366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:12.977466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1863b3-1f9b-430d-91c0-7dbed062366a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:13.049080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2f182-ea11-437c-a963-5126bc36f905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:13.049678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2f182-ea11-437c-a963-5126bc36f905 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:13.049953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2f182-ea11-437c-a963-5126bc36f905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:13.050216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2f182-ea11-437c-a963-5126bc36f905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:13.051828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8af2f182-ea11-437c-a963-5126bc36f905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:13.058788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:13.159123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2f182-ea11-437c-a963-5126bc36f905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:13.160466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2f182-ea11-437c-a963-5126bc36f905 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:13.178256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 657/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:15.959336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a381038-79c2-4008-a291-0c54218bf753 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:15.959874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a381038-79c2-4008-a291-0c54218bf753 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:15.959874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a381038-79c2-4008-a291-0c54218bf753 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:15.961218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 666/1323] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:15.966998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97861146-cce2-457c-af25-0c7f6875f61b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:15.967423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97861146-cce2-457c-af25-0c7f6875f61b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:15.967564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97861146-cce2-457c-af25-0c7f6875f61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:16.042753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-97861146-cce2-457c-af25-0c7f6875f61b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:16.044096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-97861146-cce2-457c-af25-0c7f6875f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:16.107441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:16.148501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 658/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:16.439147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a381038-79c2-4008-a291-0c54218bf753 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:16.440719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1a381038-79c2-4008-a291-0c54218bf753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:16.543214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a381038-79c2-4008-a291-0c54218bf753 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:16.556786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:16.617132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb08fd8-0fdc-4a15-b7e6-f0ad8693ede5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:16.617731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08fd8-0fdc-4a15-b7e6-f0ad8693ede5 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:16.618010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08fd8-0fdc-4a15-b7e6-f0ad8693ede5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:16.618273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08fd8-0fdc-4a15-b7e6-f0ad8693ede5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:16.619872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ebb08fd8-0fdc-4a15-b7e6-f0ad8693ede5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:16.726754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb08fd8-0fdc-4a15-b7e6-f0ad8693ede5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:16.728122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb08fd8-0fdc-4a15-b7e6-f0ad8693ede5 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:16.746449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 667/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:18.947498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2f8f9-ddfa-4a84-917f-a2e75f8c677f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:18.948026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2f8f9-ddfa-4a84-917f-a2e75f8c677f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:18.948148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2f8f9-ddfa-4a84-917f-a2e75f8c677f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:18.949422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 659/1326] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:18.954624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067d4e4-6f4c-4426-94d4-1396d788aa8b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:18.955047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067d4e4-6f4c-4426-94d4-1396d788aa8b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:18.955208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067d4e4-6f4c-4426-94d4-1396d788aa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:19.004556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:19.023220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-4067d4e4-6f4c-4426-94d4-1396d788aa8b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:19.024356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4067d4e4-6f4c-4426-94d4-1396d788aa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:19.131919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 668/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:19.413368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2f8f9-ddfa-4a84-917f-a2e75f8c677f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:19.414976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a7c2f8f9-ddfa-4a84-917f-a2e75f8c677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:19.519002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2f8f9-ddfa-4a84-917f-a2e75f8c677f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:19.593409 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e2d53-3876-417a-a63d-f5c8c738014c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:19.594009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e2d53-3876-417a-a63d-f5c8c738014c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:19.594285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e2d53-3876-417a-a63d-f5c8c738014c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:19.594555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e2d53-3876-417a-a63d-f5c8c738014c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:19.596167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e22e2d53-3876-417a-a63d-f5c8c738014c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:19.613298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:19.704978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e2d53-3876-417a-a63d-f5c8c738014c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:19.706404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e2d53-3876-417a-a63d-f5c8c738014c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:19.724919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 660/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:21.940547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f4ea4-dddc-40c3-825e-1cd1af8a1794 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:21.941062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f4ea4-dddc-40c3-825e-1cd1af8a1794 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:21.941062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f4ea4-dddc-40c3-825e-1cd1af8a1794 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:21.942383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 669/1329] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:21.947782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dc176-1191-49aa-abe2-6a496a856e09 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:21.948211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dc176-1191-49aa-abe2-6a496a856e09 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:21.948338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dc176-1191-49aa-abe2-6a496a856e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:22.014460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-607dc176-1191-49aa-abe2-6a496a856e09 None None] MFA Rules not processed for user `4dbfa2988ba9444197b16560d6e65096`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:22.015642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-607dc176-1191-49aa-abe2-6a496a856e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:22.080282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:22.112383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 661/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:22.398518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f4ea4-dddc-40c3-825e-1cd1af8a1794 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:22.400327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-466f4ea4-dddc-40c3-825e-1cd1af8a1794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:22.505067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f4ea4-dddc-40c3-825e-1cd1af8a1794 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:22.525744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:22.579021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad01d75-b41c-44e2-a867-23203781a52c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:22.579629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad01d75-b41c-44e2-a867-23203781a52c service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:22.579909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad01d75-b41c-44e2-a867-23203781a52c service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:22.580178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad01d75-b41c-44e2-a867-23203781a52c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:22.581741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6ad01d75-b41c-44e2-a867-23203781a52c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:22.688613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad01d75-b41c-44e2-a867-23203781a52c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:22.690042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad01d75-b41c-44e2-a867-23203781a52c service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:22.708268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 670/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:25.231409 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65586187-1ed0-46aa-b02e-346d5a886c58 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:25.231945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65586187-1ed0-46aa-b02e-346d5a886c58 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:25.231945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65586187-1ed0-46aa-b02e-346d5a886c58 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:25.233296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 662/1332] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:25.238366 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9e2b9-9e2d-4aca-8784-77e4ddd29375 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:25.238768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9e2b9-9e2d-4aca-8784-77e4ddd29375 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:25.238953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9e2b9-9e2d-4aca-8784-77e4ddd29375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:25.302494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:25.305057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-63a9e2b9-9e2d-4aca-8784-77e4ddd29375 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:25.306964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-63a9e2b9-9e2d-4aca-8784-77e4ddd29375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:25.401550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 671/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:51:25.681566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65586187-1ed0-46aa-b02e-346d5a886c58 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:25.683689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-65586187-1ed0-46aa-b02e-346d5a886c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:25.784335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65586187-1ed0-46aa-b02e-346d5a886c58 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:25.855531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2804f5-5c4d-4817-8c49-11e7e0bc10dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:25.856124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2804f5-5c4d-4817-8c49-11e7e0bc10dc service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:25.856402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2804f5-5c4d-4817-8c49-11e7e0bc10dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:25.856672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2804f5-5c4d-4817-8c49-11e7e0bc10dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:25.858238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8c2804f5-5c4d-4817-8c49-11e7e0bc10dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:25.884459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:25.964957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2804f5-5c4d-4817-8c49-11e7e0bc10dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:25.966311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2804f5-5c4d-4817-8c49-11e7e0bc10dc service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:25.984263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 663/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:28.197741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9604-c7e2-43fa-b93c-bf65eb986ff0 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:28.198392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9604-c7e2-43fa-b93c-bf65eb986ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:28.198392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9604-c7e2-43fa-b93c-bf65eb986ff0 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:28.199641 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 672/1335] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:28.204741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09943-24b7-48cb-a0d6-32ea65427370 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:28.205177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09943-24b7-48cb-a0d6-32ea65427370 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:28.205285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f09943-24b7-48cb-a0d6-32ea65427370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:28.270670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a5f09943-24b7-48cb-a0d6-32ea65427370 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:28.271890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a5f09943-24b7-48cb-a0d6-32ea65427370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:28.345424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:28.368412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 664/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:28.663706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b9604-c7e2-43fa-b93c-bf65eb986ff0 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:28.665257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4b7b9604-c7e2-43fa-b93c-bf65eb986ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:28.768058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b9604-c7e2-43fa-b93c-bf65eb986ff0 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:28.795662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:28.842148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d836b-a3a2-43a8-ab15-b5c5278f48b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:28.842745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d836b-a3a2-43a8-ab15-b5c5278f48b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:28.843056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d836b-a3a2-43a8-ab15-b5c5278f48b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:28.843332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d836b-a3a2-43a8-ab15-b5c5278f48b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:28.844910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-833d836b-a3a2-43a8-ab15-b5c5278f48b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:28.950380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d836b-a3a2-43a8-ab15-b5c5278f48b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:28.951759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d836b-a3a2-43a8-ab15-b5c5278f48b0 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:28.969600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 673/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:31.578701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29dde5-cb00-4c5e-8d3d-682aca9aad08 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:31.579252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29dde5-cb00-4c5e-8d3d-682aca9aad08 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:31.579370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29dde5-cb00-4c5e-8d3d-682aca9aad08 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:31.580628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 665/1338] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:31.585716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61ad9e-2a8d-4195-a927-efada0a133ca None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:31.586154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61ad9e-2a8d-4195-a927-efada0a133ca None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:31.586257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61ad9e-2a8d-4195-a927-efada0a133ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:31.650653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6c61ad9e-2a8d-4195-a927-efada0a133ca None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:31.651902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6c61ad9e-2a8d-4195-a927-efada0a133ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:31.659889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:31.748517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 674/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:32.036828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29dde5-cb00-4c5e-8d3d-682aca9aad08 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:32.038374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bc29dde5-cb00-4c5e-8d3d-682aca9aad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:32.140477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29dde5-cb00-4c5e-8d3d-682aca9aad08 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:32.211242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfcbe3-bee1-40cc-af80-6275c5aede4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:32.211842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfcbe3-bee1-40cc-af80-6275c5aede4c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:32.212121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfcbe3-bee1-40cc-af80-6275c5aede4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:32.212389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfcbe3-bee1-40cc-af80-6275c5aede4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:32.214010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25dfcbe3-bee1-40cc-af80-6275c5aede4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:32.249334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:32.321451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfcbe3-bee1-40cc-af80-6275c5aede4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:32.322875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfcbe3-bee1-40cc-af80-6275c5aede4c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:32.340851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 666/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:36.445205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339395-d362-47a1-b6c6-bac13701c72a None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:36.445773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339395-d362-47a1-b6c6-bac13701c72a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:36.445878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339395-d362-47a1-b6c6-bac13701c72a None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:36.447210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 675/1341] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:36.453104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d047f6-3dd4-4448-88d1-582336354731 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:36.453532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d047f6-3dd4-4448-88d1-582336354731 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:36.453664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d047f6-3dd4-4448-88d1-582336354731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:36.519851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c6d047f6-3dd4-4448-88d1-582336354731 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:36.520963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c6d047f6-3dd4-4448-88d1-582336354731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:36.607590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:36.619016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 667/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:51:36.904367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be339395-d362-47a1-b6c6-bac13701c72a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:36.905896 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-be339395-d362-47a1-b6c6-bac13701c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:37.008901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be339395-d362-47a1-b6c6-bac13701c72a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:37.047213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:37.082940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85c4b5-d16f-4c27-82e7-9338f4c366a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:37.083540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85c4b5-d16f-4c27-82e7-9338f4c366a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:37.083818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85c4b5-d16f-4c27-82e7-9338f4c366a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:37.084090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85c4b5-d16f-4c27-82e7-9338f4c366a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:37.085657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-eb85c4b5-d16f-4c27-82e7-9338f4c366a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:37.195084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85c4b5-d16f-4c27-82e7-9338f4c366a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:37.196436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85c4b5-d16f-4c27-82e7-9338f4c366a8 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:37.215575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 676/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:39.963061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9786-5f20-4f14-89ff-3686e71b23e3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:39.964226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9786-5f20-4f14-89ff-3686e71b23e3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:39.964226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9786-5f20-4f14-89ff-3686e71b23e3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:39.965459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 668/1344] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:39.970698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9b3c-4261-46a4-8968-c280603c6618 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:39.971151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9b3c-4261-46a4-8968-c280603c6618 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:39.971289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9b3c-4261-46a4-8968-c280603c6618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:40.036384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-95ca9b3c-4261-46a4-8968-c280603c6618 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:40.037608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-95ca9b3c-4261-46a4-8968-c280603c6618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:40.055728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:40.136988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 677/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:40.414458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9786-5f20-4f14-89ff-3686e71b23e3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:40.416048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-50ea9786-5f20-4f14-89ff-3686e71b23e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:40.522471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9786-5f20-4f14-89ff-3686e71b23e3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:40.601949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9396954-41f5-424b-8ae2-0c793b705ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:40.602551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9396954-41f5-424b-8ae2-0c793b705ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:40.602829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9396954-41f5-424b-8ae2-0c793b705ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:40.603128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9396954-41f5-424b-8ae2-0c793b705ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:40.604736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e9396954-41f5-424b-8ae2-0c793b705ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:40.648063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:40.716017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9396954-41f5-424b-8ae2-0c793b705ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:40.717361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9396954-41f5-424b-8ae2-0c793b705ab4 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:40.735588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 669/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:44.813630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aa4db-f22c-4f2e-ad31-5a07eb1b852e None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:44.814231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aa4db-f22c-4f2e-ad31-5a07eb1b852e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:44.814231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aa4db-f22c-4f2e-ad31-5a07eb1b852e None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:44.815685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 678/1347] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:44.821016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37527-d443-4ae4-a85f-b5a6660e7f00 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:44.821436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37527-d443-4ae4-a85f-b5a6660e7f00 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:44.821571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37527-d443-4ae4-a85f-b5a6660e7f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:44.889313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9ee37527-d443-4ae4-a85f-b5a6660e7f00 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:44.890579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9ee37527-d443-4ae4-a85f-b5a6660e7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:44.989448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:44.992002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 670/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:45.274548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08aa4db-f22c-4f2e-ad31-5a07eb1b852e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:45.276134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e08aa4db-f22c-4f2e-ad31-5a07eb1b852e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:45.382757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08aa4db-f22c-4f2e-ad31-5a07eb1b852e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:45.435797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:45.462491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7ee68-5e82-46cb-873a-09b0462e2b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:45.463156 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7ee68-5e82-46cb-873a-09b0462e2b2f service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:45.463428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7ee68-5e82-46cb-873a-09b0462e2b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:45.463690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7ee68-5e82-46cb-873a-09b0462e2b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:45.465267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-11d7ee68-5e82-46cb-873a-09b0462e2b2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:45.577419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7ee68-5e82-46cb-873a-09b0462e2b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:45.578858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7ee68-5e82-46cb-873a-09b0462e2b2f service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:45.597485 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 679/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:51:50.556219 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f8b36-9a24-49ba-8a11-81842a87188e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:50.557207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f8b36-9a24-49ba-8a11-81842a87188e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:50.557302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f8b36-9a24-49ba-8a11-81842a87188e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:50.559037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 671/1350] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 11:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:50.563984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c282e-527b-4110-bbbb-83fa4e892a7f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:50.564383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c282e-527b-4110-bbbb-83fa4e892a7f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:50.564551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c282e-527b-4110-bbbb-83fa4e892a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:50.675040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-125c282e-527b-4110-bbbb-83fa4e892a7f None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:50.676693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-125c282e-527b-4110-bbbb-83fa4e892a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:50.696644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:50.836372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 680/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 11:51:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:51:54.877085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045822d9-59aa-41f9-9a10-89914e356afa None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:54.877616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045822d9-59aa-41f9-9a10-89914e356afa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:54.877616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045822d9-59aa-41f9-9a10-89914e356afa None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:54.879137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 672/1352] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 11:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:54.883974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4feb34-1ec4-4f20-a1a8-f0d81fee4130 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:54.884439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4feb34-1ec4-4f20-a1a8-f0d81fee4130 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:54.884555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4feb34-1ec4-4f20-a1a8-f0d81fee4130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:54.964049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6c4feb34-1ec4-4f20-a1a8-f0d81fee4130 None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:54.965483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6c4feb34-1ec4-4f20-a1a8-f0d81fee4130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:55.040541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:55.078856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 681/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 11:51:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:51:56.619489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278b40d-1002-4e1e-8bfe-3ce404dff1b9 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:56.620004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278b40d-1002-4e1e-8bfe-3ce404dff1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:56.620097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278b40d-1002-4e1e-8bfe-3ce404dff1b9 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:56.621387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 673/1354] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 11:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:56.626473 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066498-f8ec-4e83-9d7d-6de2cae8bec0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:56.626881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066498-f8ec-4e83-9d7d-6de2cae8bec0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:56.627166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066498-f8ec-4e83-9d7d-6de2cae8bec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:56.704274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-96066498-f8ec-4e83-9d7d-6de2cae8bec0 None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:56.705387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-96066498-f8ec-4e83-9d7d-6de2cae8bec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:56.811331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 682/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 11:51:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:00.272889 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d8a38-fe97-4e21-9737-95dde14e54a5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:00.273426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d8a38-fe97-4e21-9737-95dde14e54a5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:00.273426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d8a38-fe97-4e21-9737-95dde14e54a5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:00.274869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 674/1356] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:00.280211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10d6b4-95fb-448b-aa6c-146963d45b43 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:00.280704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10d6b4-95fb-448b-aa6c-146963d45b43 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:00.280704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10d6b4-95fb-448b-aa6c-146963d45b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:00.323640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:00.360886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5e10d6b4-95fb-448b-aa6c-146963d45b43 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:00.362006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5e10d6b4-95fb-448b-aa6c-146963d45b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:00.469572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 683/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:00.767420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea882-8d21-43ee-86ba-2a3696c8d956 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:00.768076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea882-8d21-43ee-86ba-2a3696c8d956 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:00.768661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea882-8d21-43ee-86ba-2a3696c8d956 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:00.770361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 675/1358] 10.222.0.52 () {54 vars in 961 bytes} [Mon Oct 10 11:52:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:00.780033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6435f-6fa5-40b9-8ef4-5db7bc2618f5 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:00.780582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6435f-6fa5-40b9-8ef4-5db7bc2618f5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:00.781155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6435f-6fa5-40b9-8ef4-5db7bc2618f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:00.872410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-7cb6435f-6fa5-40b9-8ef4-5db7bc2618f5 None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:00.873964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7cb6435f-6fa5-40b9-8ef4-5db7bc2618f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:00.926646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:01.003122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 684/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Oct 10 11:52:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:52:01.008098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea882-8d21-43ee-86ba-2a3696c8d956 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:01.009882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-820ea882-8d21-43ee-86ba-2a3696c8d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:01.173350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea882-8d21-43ee-86ba-2a3696c8d956 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:01.254040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:01.297907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cb540-94d1-4037-9d45-db4721e4b8ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:01.298562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cb540-94d1-4037-9d45-db4721e4b8ab service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:01.298842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cb540-94d1-4037-9d45-db4721e4b8ab service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:01.299252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cb540-94d1-4037-9d45-db4721e4b8ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:01.300980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9b8cb540-94d1-4037-9d45-db4721e4b8ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:01.415507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cb540-94d1-4037-9d45-db4721e4b8ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:01.417507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cb540-94d1-4037-9d45-db4721e4b8ab service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:01.437292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 676/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:03.817464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec75b-930c-41d8-a51e-bab7d6d8bd58 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:03.817464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec75b-930c-41d8-a51e-bab7d6d8bd58 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:03.818354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec75b-930c-41d8-a51e-bab7d6d8bd58 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:03.825182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10451c0d-8fae-41ac-af82-668bf0657711 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:03.825182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10451c0d-8fae-41ac-af82-668bf0657711 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:03.825182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10451c0d-8fae-41ac-af82-668bf0657711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:03.830007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 685/1361] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:03 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:03.898034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-10451c0d-8fae-41ac-af82-668bf0657711 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:03.899400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10451c0d-8fae-41ac-af82-668bf0657711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:03.938858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:03.999544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 677/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:52:04.291243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ec75b-930c-41d8-a51e-bab7d6d8bd58 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:04.292768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f87ec75b-930c-41d8-a51e-bab7d6d8bd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:04.465296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ec75b-930c-41d8-a51e-bab7d6d8bd58 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:04.476908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:04.610358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc543b-1c84-4f59-8ee3-62e37f06dfd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:04.610997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc543b-1c84-4f59-8ee3-62e37f06dfd7 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:04.611278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc543b-1c84-4f59-8ee3-62e37f06dfd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:04.611550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc543b-1c84-4f59-8ee3-62e37f06dfd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:04.613146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-12cc543b-1c84-4f59-8ee3-62e37f06dfd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:04.705409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc543b-1c84-4f59-8ee3-62e37f06dfd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:04.707192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc543b-1c84-4f59-8ee3-62e37f06dfd7 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:04.726329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 686/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:07.236722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ade8b-e3f0-484e-8ce6-16700bc181ca None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:07.237273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ade8b-e3f0-484e-8ce6-16700bc181ca None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:07.237273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ade8b-e3f0-484e-8ce6-16700bc181ca None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:07.238787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 678/1364] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:07.243960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95910d86-8fb7-489b-9373-c15cfed91b6d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:07.244418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95910d86-8fb7-489b-9373-c15cfed91b6d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:07.244501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95910d86-8fb7-489b-9373-c15cfed91b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:07.261011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:07.314292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-95910d86-8fb7-489b-9373-c15cfed91b6d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:07.315448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-95910d86-8fb7-489b-9373-c15cfed91b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:07.425215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 687/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:52:07.726603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200450-b5f6-49e1-802d-54e353b9262f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:07.727048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200450-b5f6-49e1-802d-54e353b9262f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:07.727447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79200450-b5f6-49e1-802d-54e353b9262f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:07.729656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 679/1366] 10.222.0.52 () {54 vars in 961 bytes} [Mon Oct 10 11:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:07.734963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7e30-8fce-4138-b27a-d3ee532032bc None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:07.735273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7e30-8fce-4138-b27a-d3ee532032bc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:07.735554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7e30-8fce-4138-b27a-d3ee532032bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:07.814468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-fe1b7e30-8fce-4138-b27a-d3ee532032bc None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:07.815693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fe1b7e30-8fce-4138-b27a-d3ee532032bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:07.852332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:07.939989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 688/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Oct 10 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:07.948362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79200450-b5f6-49e1-802d-54e353b9262f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:07.950232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-79200450-b5f6-49e1-802d-54e353b9262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:08.105886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79200450-b5f6-49e1-802d-54e353b9262f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:08.126648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:08.225981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6c58b-20c3-4522-b226-e4362dede07e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:08.226653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6c58b-20c3-4522-b226-e4362dede07e service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:08.226962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6c58b-20c3-4522-b226-e4362dede07e service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:08.227239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6c58b-20c3-4522-b226-e4362dede07e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:08.228891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-93b6c58b-20c3-4522-b226-e4362dede07e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:08.329309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6c58b-20c3-4522-b226-e4362dede07e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:08.331121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6c58b-20c3-4522-b226-e4362dede07e service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:08.351757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 680/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:08.422219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76de17-b4df-4145-b8da-75a544893e0e None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:08.422743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76de17-b4df-4145-b8da-75a544893e0e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:08.422743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76de17-b4df-4145-b8da-75a544893e0e None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:08.426673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 689/1369] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:08.429407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f917ee-5bdb-42cb-83c0-af53588a848f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:08.429723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f917ee-5bdb-42cb-83c0-af53588a848f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:08.430005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f917ee-5bdb-42cb-83c0-af53588a848f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:08.468691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:08.522658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-06f917ee-5bdb-42cb-83c0-af53588a848f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:08.523964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-06f917ee-5bdb-42cb-83c0-af53588a848f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:08.641776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 681/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:52:08.993958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19bc02-8e55-4d82-90e7-834505fc58da None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:08.994461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19bc02-8e55-4d82-90e7-834505fc58da None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:08.994878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19bc02-8e55-4d82-90e7-834505fc58da None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:08.996377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 690/1371] 10.222.0.52 () {54 vars in 963 bytes} [Mon Oct 10 11:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:09.001486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a62c6-1b3a-4003-ab13-95110a2ec795 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:09.001949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a62c6-1b3a-4003-ab13-95110a2ec795 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:09.002369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a62c6-1b3a-4003-ab13-95110a2ec795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:09.083497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c26a62c6-1b3a-4003-ab13-95110a2ec795 None None] MFA Rules not processed for user `75aa23cd162e4da2bd6a936c27e51f41`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:09.084802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c26a62c6-1b3a-4003-ab13-95110a2ec795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:09.139082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:09.172565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 682/1372] 10.222.0.52 () {58 vars in 1060 bytes} [Mon Oct 10 11:52:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:09.180494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19bc02-8e55-4d82-90e7-834505fc58da None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:09.182054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3d19bc02-8e55-4d82-90e7-834505fc58da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:09.294767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19bc02-8e55-4d82-90e7-834505fc58da None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:09.343212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:09.381273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc8e9-5974-494f-92dd-5224eb8cf618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:09.381878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc8e9-5974-494f-92dd-5224eb8cf618 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:09.382193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc8e9-5974-494f-92dd-5224eb8cf618 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:09.382408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc8e9-5974-494f-92dd-5224eb8cf618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:09.384742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c0fcc8e9-5974-494f-92dd-5224eb8cf618 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:09.539141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc8e9-5974-494f-92dd-5224eb8cf618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:09.541147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc8e9-5974-494f-92dd-5224eb8cf618 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:09.561039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 691/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:52:10.614263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106fdd3-d18d-4fea-bba5-429ad2b1fc52 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:10.614794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106fdd3-d18d-4fea-bba5-429ad2b1fc52 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:10.614957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106fdd3-d18d-4fea-bba5-429ad2b1fc52 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:10.616311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 683/1374] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:10.621535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eec181-9a43-46b8-ac97-c77f2a6a8485 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:10.621951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eec181-9a43-46b8-ac97-c77f2a6a8485 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:10.622088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eec181-9a43-46b8-ac97-c77f2a6a8485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:10.686823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-31eec181-9a43-46b8-ac97-c77f2a6a8485 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:10.687979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-31eec181-9a43-46b8-ac97-c77f2a6a8485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:10.707983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:10.785386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 692/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:11.077391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106fdd3-d18d-4fea-bba5-429ad2b1fc52 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:11.078961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2106fdd3-d18d-4fea-bba5-429ad2b1fc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:11.234762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106fdd3-d18d-4fea-bba5-429ad2b1fc52 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:11.289529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:11.357811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25129ad-6e1a-4e99-a1a7-8538681956eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:11.358411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25129ad-6e1a-4e99-a1a7-8538681956eb service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:11.358691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25129ad-6e1a-4e99-a1a7-8538681956eb service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:11.358987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25129ad-6e1a-4e99-a1a7-8538681956eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:11.360596 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f25129ad-6e1a-4e99-a1a7-8538681956eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:11.457867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25129ad-6e1a-4e99-a1a7-8538681956eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:11.459544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25129ad-6e1a-4e99-a1a7-8538681956eb service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:11.478805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 684/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:52:13.670572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211fd1f-9338-4798-85bc-8ddb2ac56d6a None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:13.671107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211fd1f-9338-4798-85bc-8ddb2ac56d6a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:13.671107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211fd1f-9338-4798-85bc-8ddb2ac56d6a None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:13.672428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 693/1377] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:13.678087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c41273-a39f-4878-8482-c37c7f3c6a9c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:13.678502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c41273-a39f-4878-8482-c37c7f3c6a9c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:13.678605 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c41273-a39f-4878-8482-c37c7f3c6a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:13.749099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-31c41273-a39f-4878-8482-c37c7f3c6a9c None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:13.750286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-31c41273-a39f-4878-8482-c37c7f3c6a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:13.756948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:13.852459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 685/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:14.145348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211fd1f-9338-4798-85bc-8ddb2ac56d6a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:14.147037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f211fd1f-9338-4798-85bc-8ddb2ac56d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:14.300026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211fd1f-9338-4798-85bc-8ddb2ac56d6a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:14.304439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:14.427866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5bb5d-2843-419b-9014-13917ce4b735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:14.428470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5bb5d-2843-419b-9014-13917ce4b735 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:14.428746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5bb5d-2843-419b-9014-13917ce4b735 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:14.429022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5bb5d-2843-419b-9014-13917ce4b735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:14.430847 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-86b5bb5d-2843-419b-9014-13917ce4b735 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:14.523225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5bb5d-2843-419b-9014-13917ce4b735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:14.525336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5bb5d-2843-419b-9014-13917ce4b735 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:14.544414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 694/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:16.778416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e83e3-ae22-47e4-ad5d-6a33fc37c6a4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:16.778913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e83e3-ae22-47e4-ad5d-6a33fc37c6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:16.779035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e83e3-ae22-47e4-ad5d-6a33fc37c6a4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:16.780485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 686/1380] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:16.785522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ff95-6c20-495d-b845-8743a30d5e5b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:16.785963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ff95-6c20-495d-b845-8743a30d5e5b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:16.786064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ff95-6c20-495d-b845-8743a30d5e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:16.798191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:16.855785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d769ff95-6c20-495d-b845-8743a30d5e5b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:16.856971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d769ff95-6c20-495d-b845-8743a30d5e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:16.971265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 695/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:17.275260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e83e3-ae22-47e4-ad5d-6a33fc37c6a4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:17.277277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-356e83e3-ae22-47e4-ad5d-6a33fc37c6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:17.435724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:17.447000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e83e3-ae22-47e4-ad5d-6a33fc37c6a4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:17.570831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ecdd3-4dfb-49ac-bde1-dc939fe3953e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:17.571231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ecdd3-4dfb-49ac-bde1-dc939fe3953e service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:17.571508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ecdd3-4dfb-49ac-bde1-dc939fe3953e service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:17.571777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ecdd3-4dfb-49ac-bde1-dc939fe3953e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:17.573377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1c9ecdd3-4dfb-49ac-bde1-dc939fe3953e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:17.664779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ecdd3-4dfb-49ac-bde1-dc939fe3953e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:17.667468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ecdd3-4dfb-49ac-bde1-dc939fe3953e service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:17.680267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:17.691059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 687/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:19.881708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1527-a631-45ba-87e9-10daffcbf655 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:19.882240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1527-a631-45ba-87e9-10daffcbf655 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:19.882240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1527-a631-45ba-87e9-10daffcbf655 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:19.884202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 696/1383] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:19.889951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9f39-35b1-46dd-8267-cbad6aa976fd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:19.890510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9f39-35b1-46dd-8267-cbad6aa976fd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:19.890863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9f39-35b1-46dd-8267-cbad6aa976fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:19.962691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f1bf9f39-35b1-46dd-8267-cbad6aa976fd None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:19.963943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f1bf9f39-35b1-46dd-8267-cbad6aa976fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:20.065745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 688/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:20.355656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff1527-a631-45ba-87e9-10daffcbf655 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:20.357887 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-41ff1527-a631-45ba-87e9-10daffcbf655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:20.450675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:20.511977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff1527-a631-45ba-87e9-10daffcbf655 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:20.639067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f88720-34bc-43fc-8521-85aa8a4412f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:20.639603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88720-34bc-43fc-8521-85aa8a4412f8 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:20.639884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88720-34bc-43fc-8521-85aa8a4412f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:20.640180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88720-34bc-43fc-8521-85aa8a4412f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:20.641785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-56f88720-34bc-43fc-8521-85aa8a4412f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:20.693489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:20.733595 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f88720-34bc-43fc-8521-85aa8a4412f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:20.735225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f88720-34bc-43fc-8521-85aa8a4412f8 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:20.752833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 697/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:23.338081 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9982d-9cdc-446f-882b-f50f6c086afe None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:23.338668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9982d-9cdc-446f-882b-f50f6c086afe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:23.338668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9982d-9cdc-446f-882b-f50f6c086afe None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:23.342386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 689/1386] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:23.345496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912365e5-babc-4098-a52c-de89c4a58a91 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:23.345927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912365e5-babc-4098-a52c-de89c4a58a91 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:23.346056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912365e5-babc-4098-a52c-de89c4a58a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:23.417378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-912365e5-babc-4098-a52c-de89c4a58a91 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:23.418606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-912365e5-babc-4098-a52c-de89c4a58a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:23.517318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 698/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:23.804427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9982d-9cdc-446f-882b-f50f6c086afe None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:23.807224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-61e9982d-9cdc-446f-882b-f50f6c086afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:23.897363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:23.970569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9982d-9cdc-446f-882b-f50f6c086afe None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:24.085400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d733d-fa4b-4baf-8af6-687fce10251a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:24.086134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d733d-fa4b-4baf-8af6-687fce10251a service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:24.086420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d733d-fa4b-4baf-8af6-687fce10251a service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:24.086694 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d733d-fa4b-4baf-8af6-687fce10251a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:24.088921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8a4d733d-fa4b-4baf-8af6-687fce10251a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:24.129648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:24.180802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d733d-fa4b-4baf-8af6-687fce10251a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:24.182819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d733d-fa4b-4baf-8af6-687fce10251a service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:24.201678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 690/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:24.796709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c462a4-82e6-4e85-b5bf-d7e60c45f27c None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:24.797285 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c462a4-82e6-4e85-b5bf-d7e60c45f27c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:24.797285 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c462a4-82e6-4e85-b5bf-d7e60c45f27c None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:24.798612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 699/1389] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:24.804076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cfd61-7ec8-4e05-afbd-f14b6e4503be None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:24.804468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cfd61-7ec8-4e05-afbd-f14b6e4503be None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:24.804624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cfd61-7ec8-4e05-afbd-f14b6e4503be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:24.872885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fc0cfd61-7ec8-4e05-afbd-f14b6e4503be None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:24.874081 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fc0cfd61-7ec8-4e05-afbd-f14b6e4503be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:24.971134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 691/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:25.270315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2799-ae53-49f4-8610-8675c1ad1c02 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:25.270606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2799-ae53-49f4-8610-8675c1ad1c02 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:25.270882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2799-ae53-49f4-8610-8675c1ad1c02 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:25.272234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 700/1391] 10.222.0.52 () {54 vars in 962 bytes} [Mon Oct 10 11:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:52:25.277385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d356-0d62-493b-a441-b710205528dc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:25.277677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d356-0d62-493b-a441-b710205528dc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:25.277956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d356-0d62-493b-a441-b710205528dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:25.294105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:25.355687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a9b9d356-0d62-493b-a441-b710205528dc None None] MFA Rules not processed for user `c6774fdd8b2e46b6a7d00919568f9fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:25.356905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a9b9d356-0d62-493b-a441-b710205528dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:25.435916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 692/1392] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Oct 10 11:52:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:25.445004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2799-ae53-49f4-8610-8675c1ad1c02 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:25.447108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2cca2799-ae53-49f4-8610-8675c1ad1c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:25.478393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:25.550729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2799-ae53-49f4-8610-8675c1ad1c02 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:25.623619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75640eea-a2bd-4592-8c19-f1510482e3e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6774fdd8b2e46b6a7d00919568f9fd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:25.624221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75640eea-a2bd-4592-8c19-f1510482e3e9 service glance] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:25.624486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75640eea-a2bd-4592-8c19-f1510482e3e9 service glance] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:25.624755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75640eea-a2bd-4592-8c19-f1510482e3e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:25.626311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-75640eea-a2bd-4592-8c19-f1510482e3e9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:25.703171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:25.735232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75640eea-a2bd-4592-8c19-f1510482e3e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:25.736702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75640eea-a2bd-4592-8c19-f1510482e3e9 service glance] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:25.755236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 701/1393] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:26.396161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ffe90-0e24-413e-b5d9-b8ddd9affe6a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:26.396749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ffe90-0e24-413e-b5d9-b8ddd9affe6a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:26.396749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ffe90-0e24-413e-b5d9-b8ddd9affe6a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:26.398053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 693/1394] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:26.403560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b19d5-f76f-4a42-9285-b5b47afaa320 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:26.404049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b19d5-f76f-4a42-9285-b5b47afaa320 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:26.404158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b19d5-f76f-4a42-9285-b5b47afaa320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:26.480064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-748b19d5-f76f-4a42-9285-b5b47afaa320 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:26.481311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-748b19d5-f76f-4a42-9285-b5b47afaa320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:26.500121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e33e9-3904-41ab-bb12-a1f77c4f8a03 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:26.500350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e33e9-3904-41ab-bb12-a1f77c4f8a03 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:26.500652 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e33e9-3904-41ab-bb12-a1f77c4f8a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:26.566732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b13e33e9-3904-41ab-bb12-a1f77c4f8a03 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:26.567962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b13e33e9-3904-41ab-bb12-a1f77c4f8a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:26.582935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 702/1395] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:52:26.624656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:26.656119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 694/1396] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:52:26.699526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9557a8-3236-42f7-8f66-f058976deadb None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:26.699865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9557a8-3236-42f7-8f66-f058976deadb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:26.700140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9557a8-3236-42f7-8f66-f058976deadb None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:26.701490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 703/1397] 10.222.0.52 () {54 vars in 943 bytes} [Mon Oct 10 11:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:26.707766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a847-e46f-415f-9998-15f9e4fcf648 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:26.708064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a847-e46f-415f-9998-15f9e4fcf648 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:26.708339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a847-e46f-415f-9998-15f9e4fcf648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:26.786166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4559a847-e46f-415f-9998-15f9e4fcf648 None None] MFA Rules not processed for user `91df91b1ea4949119245c86bf52e32f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:26.787353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4559a847-e46f-415f-9998-15f9e4fcf648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:26.869035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9557a8-3236-42f7-8f66-f058976deadb None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:26.870548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ea9557a8-3236-42f7-8f66-f058976deadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:26.895827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 695/1398] 10.222.0.52 () {58 vars in 1040 bytes} [Mon Oct 10 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:52:26.905581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559a847-e46f-415f-9998-15f9e4fcf648 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:26.912449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:26.916110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559a847-e46f-415f-9998-15f9e4fcf648 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:26.942287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:27.019648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9557a8-3236-42f7-8f66-f058976deadb None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:27.021468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28902f60-3e09-4b91-8cb7-5adb4f211a60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:27.022077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28902f60-3e09-4b91-8cb7-5adb4f211a60 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:27.022467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28902f60-3e09-4b91-8cb7-5adb4f211a60 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:27.022625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28902f60-3e09-4b91-8cb7-5adb4f211a60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:27.024298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-28902f60-3e09-4b91-8cb7-5adb4f211a60 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:27.109586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28902f60-3e09-4b91-8cb7-5adb4f211a60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:27.111203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28902f60-3e09-4b91-8cb7-5adb4f211a60 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:27.119261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 696/1399] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 11:52:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 11:52:27.135554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:27.138862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925d6fb-a692-49b2-8754-ae0fb770e2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:27.139679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925d6fb-a692-49b2-8754-ae0fb770e2bc service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:27.140139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925d6fb-a692-49b2-8754-ae0fb770e2bc service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:27.140622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925d6fb-a692-49b2-8754-ae0fb770e2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:27.142440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9925d6fb-a692-49b2-8754-ae0fb770e2bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:27.245040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925d6fb-a692-49b2-8754-ae0fb770e2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:27.246858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925d6fb-a692-49b2-8754-ae0fb770e2bc service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:27.267571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 704/1400] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:29.527086 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e85fe-91bb-4536-9c6e-ae5b8468e025 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:29.527649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e85fe-91bb-4536-9c6e-ae5b8468e025 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:29.527649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e85fe-91bb-4536-9c6e-ae5b8468e025 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:29.528962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 697/1401] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:29.534207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35269c23-8fe9-4001-94da-7fddfd29fe39 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:29.535229 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35269c23-8fe9-4001-94da-7fddfd29fe39 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:29.535508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35269c23-8fe9-4001-94da-7fddfd29fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:29.613392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-35269c23-8fe9-4001-94da-7fddfd29fe39 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:29.615246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-35269c23-8fe9-4001-94da-7fddfd29fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:29.690111 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:29.721702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 705/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:30.014763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e85fe-91bb-4536-9c6e-ae5b8468e025 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:30.016382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a44e85fe-91bb-4536-9c6e-ae5b8468e025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:30.081712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:30.181117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e85fe-91bb-4536-9c6e-ae5b8468e025 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:30.310603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553eb51c-f204-4c8d-9ae2-5527fd81cecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:30.311584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553eb51c-f204-4c8d-9ae2-5527fd81cecd service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:30.311918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553eb51c-f204-4c8d-9ae2-5527fd81cecd service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:30.312249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553eb51c-f204-4c8d-9ae2-5527fd81cecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:30.313957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-553eb51c-f204-4c8d-9ae2-5527fd81cecd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:30.334628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:30.443710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553eb51c-f204-4c8d-9ae2-5527fd81cecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:30.447404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553eb51c-f204-4c8d-9ae2-5527fd81cecd service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:30.478474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 698/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:32.710110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267b7fe-b40c-42f0-8d95-61a97e9389de None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:32.710708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267b7fe-b40c-42f0-8d95-61a97e9389de None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:32.710708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267b7fe-b40c-42f0-8d95-61a97e9389de None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:32.715234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 706/1404] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:32.717210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62365fe3-237a-42df-ad65-e5dc579f6a26 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:32.717660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62365fe3-237a-42df-ad65-e5dc579f6a26 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:32.717749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62365fe3-237a-42df-ad65-e5dc579f6a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:32.787766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-62365fe3-237a-42df-ad65-e5dc579f6a26 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:32.788989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-62365fe3-237a-42df-ad65-e5dc579f6a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:32.891552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:32.898400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 699/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:33.189623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267b7fe-b40c-42f0-8d95-61a97e9389de None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:33.191186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6267b7fe-b40c-42f0-8d95-61a97e9389de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:33.347174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267b7fe-b40c-42f0-8d95-61a97e9389de None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:33.409387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:33.473520 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f99ce8-7c20-4b0c-bc96-d8a1d06505bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:33.474033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f99ce8-7c20-4b0c-bc96-d8a1d06505bd service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:33.474312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f99ce8-7c20-4b0c-bc96-d8a1d06505bd service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:33.474580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f99ce8-7c20-4b0c-bc96-d8a1d06505bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:33.476221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-89f99ce8-7c20-4b0c-bc96-d8a1d06505bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:33.583236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f99ce8-7c20-4b0c-bc96-d8a1d06505bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:33.584976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f99ce8-7c20-4b0c-bc96-d8a1d06505bd service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:33.608101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 707/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:35.841654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7dab5-222d-4317-a3bf-aa1060b2d24b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:35.842221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7dab5-222d-4317-a3bf-aa1060b2d24b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:35.842221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7dab5-222d-4317-a3bf-aa1060b2d24b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:35.843771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 700/1407] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:35.849092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbca6c-65c5-49bd-bafa-03f0e97a4ae9 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:35.849580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbca6c-65c5-49bd-bafa-03f0e97a4ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:35.849580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbca6c-65c5-49bd-bafa-03f0e97a4ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:35.920369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6cdbca6c-65c5-49bd-bafa-03f0e97a4ae9 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:35.921606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6cdbca6c-65c5-49bd-bafa-03f0e97a4ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:35.936589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:36.036556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 708/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:36.330959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7dab5-222d-4317-a3bf-aa1060b2d24b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:36.333384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-75c7dab5-222d-4317-a3bf-aa1060b2d24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:36.507564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7dab5-222d-4317-a3bf-aa1060b2d24b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:36.581777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:36.634913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693321f-aae7-4428-9798-1d0aaa3e7854 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:36.635627 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693321f-aae7-4428-9798-1d0aaa3e7854 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:36.635912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693321f-aae7-4428-9798-1d0aaa3e7854 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:36.636185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693321f-aae7-4428-9798-1d0aaa3e7854 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:36.637819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a693321f-aae7-4428-9798-1d0aaa3e7854 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:36.738018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693321f-aae7-4428-9798-1d0aaa3e7854 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:36.739571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693321f-aae7-4428-9798-1d0aaa3e7854 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:36.758568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 701/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:46.034922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=38 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:46.756403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=64 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:49.973375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1110d37-c965-43c7-9a28-c4c8feae0410 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:49.973954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1110d37-c965-43c7-9a28-c4c8feae0410 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:49.974243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1110d37-c965-43c7-9a28-c4c8feae0410 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:49.975702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 709/1410] 10.222.0.52 () {54 vars in 918 bytes} [Mon Oct 10 11:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:49.981053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775bc7f-2fbf-4685-a39b-d1372eb087db None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:49.981499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775bc7f-2fbf-4685-a39b-d1372eb087db None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:49.981601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775bc7f-2fbf-4685-a39b-d1372eb087db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:50.056066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e775bc7f-2fbf-4685-a39b-d1372eb087db None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:50.057271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e775bc7f-2fbf-4685-a39b-d1372eb087db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:50.161202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 702/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Mon Oct 10 11:52:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:50.175545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b140061-449f-470c-b091-b2d3ed72948f None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:50.175829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b140061-449f-470c-b091-b2d3ed72948f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:50.176102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b140061-449f-470c-b091-b2d3ed72948f None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:50.177411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 710/1412] 10.222.0.52 () {54 vars in 918 bytes} [Mon Oct 10 11:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:50.182141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8f72-d02b-4ae6-93d5-da2709d86aef None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:50.182429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8f72-d02b-4ae6-93d5-da2709d86aef None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:50.182723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8f72-d02b-4ae6-93d5-da2709d86aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:50.255063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:50.257681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-89dd8f72-d02b-4ae6-93d5-da2709d86aef None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:50.259516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-89dd8f72-d02b-4ae6-93d5-da2709d86aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:50.374519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 703/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Mon Oct 10 11:52:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:52.925599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea36fe-1760-4cc3-b24e-450e960a10cd None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:52.926138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea36fe-1760-4cc3-b24e-450e960a10cd None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:52.926138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea36fe-1760-4cc3-b24e-450e960a10cd None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:52.927581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:52.933602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbcb63-e77b-4ffa-bae7-dc012fb3b650 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:52.934023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbcb63-e77b-4ffa-bae7-dc012fb3b650 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:52.934140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbcb63-e77b-4ffa-bae7-dc012fb3b650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:52.999157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-43dbcb63-e77b-4ffa-bae7-dc012fb3b650 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:53.000333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-43dbcb63-e77b-4ffa-bae7-dc012fb3b650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:53.053685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:53.096106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:53.381868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea36fe-1760-4cc3-b24e-450e960a10cd None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:53.383513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c9ea36fe-1760-4cc3-b24e-450e960a10cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:53.529710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea36fe-1760-4cc3-b24e-450e960a10cd None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:53.601697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:53.641668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f0d3e-b9a0-4b4d-ae0b-3e25875fef73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:53.642199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0d3e-b9a0-4b4d-ae0b-3e25875fef73 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:53.642481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0d3e-b9a0-4b4d-ae0b-3e25875fef73 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:53.642754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0d3e-b9a0-4b4d-ae0b-3e25875fef73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:53.644378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d04f0d3e-b9a0-4b4d-ae0b-3e25875fef73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:53.732878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f0d3e-b9a0-4b4d-ae0b-3e25875fef73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:53.734465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f0d3e-b9a0-4b4d-ae0b-3e25875fef73 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:53.751999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 712/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:54.889058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91965-9420-4996-b543-352db0f3675c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:54.889682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91965-9420-4996-b543-352db0f3675c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:54.889682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91965-9420-4996-b543-352db0f3675c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:54.890994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:54.897074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ffb7-9e10-4dbb-9f6e-05cb2ffcde67 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:54.897483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ffb7-9e10-4dbb-9f6e-05cb2ffcde67 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:54.897620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ffb7-9e10-4dbb-9f6e-05cb2ffcde67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:54.972551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3d89ffb7-9e10-4dbb-9f6e-05cb2ffcde67 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:54.973741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3d89ffb7-9e10-4dbb-9f6e-05cb2ffcde67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:55.000215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:55.070295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:55.221737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd91965-9420-4996-b543-352db0f3675c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:55.223181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fcd91965-9420-4996-b543-352db0f3675c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:55.365344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd91965-9420-4996-b543-352db0f3675c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:55.405789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:55.478982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb5115-fa19-4a6f-8607-3a1d51fb0eb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:55.479496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb5115-fa19-4a6f-8607-3a1d51fb0eb6 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:55.479780 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb5115-fa19-4a6f-8607-3a1d51fb0eb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:55.480955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb5115-fa19-4a6f-8607-3a1d51fb0eb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:55.481629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9cfb5115-fa19-4a6f-8607-3a1d51fb0eb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:55.570346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb5115-fa19-4a6f-8607-3a1d51fb0eb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:55.572024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb5115-fa19-4a6f-8607-3a1d51fb0eb6 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:55.590440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 706/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:56.713853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ea050-ce3f-4c9e-9571-c97eeca1c4b4 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:56.714386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ea050-ce3f-4c9e-9571-c97eeca1c4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:56.714386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ea050-ce3f-4c9e-9571-c97eeca1c4b4 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:56.715716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:56.721757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef718-0ff8-45ff-b26f-9aa1fef5a31d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:56.722185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef718-0ff8-45ff-b26f-9aa1fef5a31d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:56.722307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef718-0ff8-45ff-b26f-9aa1fef5a31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:56.775590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:56.798028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-25fef718-0ff8-45ff-b26f-9aa1fef5a31d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:56.799270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25fef718-0ff8-45ff-b26f-9aa1fef5a31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:56.893943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:57.046708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ea050-ce3f-4c9e-9571-c97eeca1c4b4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:57.048199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1f4ea050-ce3f-4c9e-9571-c97eeca1c4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:57.192969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ea050-ce3f-4c9e-9571-c97eeca1c4b4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:57.204772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:57.302834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc374b1f-1519-4472-a81c-5016f3d871aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:57.303392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc374b1f-1519-4472-a81c-5016f3d871aa service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:57.303671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc374b1f-1519-4472-a81c-5016f3d871aa service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:57.303940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc374b1f-1519-4472-a81c-5016f3d871aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:57.305805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fc374b1f-1519-4472-a81c-5016f3d871aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:57.393419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc374b1f-1519-4472-a81c-5016f3d871aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:57.395064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc374b1f-1519-4472-a81c-5016f3d871aa service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:57.413943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 715/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:52:58.523309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09170adb-d3fc-4009-b5ba-99a22d130555 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:58.523872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09170adb-d3fc-4009-b5ba-99a22d130555 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:58.523872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09170adb-d3fc-4009-b5ba-99a22d130555 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:58.525248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:52:58.531418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf684c2-6c8f-4cf9-bb4e-9460268ef693 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:58.531860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf684c2-6c8f-4cf9-bb4e-9460268ef693 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:58.531971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf684c2-6c8f-4cf9-bb4e-9460268ef693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:58.551900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:58.608383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-abf684c2-6c8f-4cf9-bb4e-9460268ef693 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:52:58.610059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-abf684c2-6c8f-4cf9-bb4e-9460268ef693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:58.823596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:52:58.972139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09170adb-d3fc-4009-b5ba-99a22d130555 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:52:58.973759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-09170adb-d3fc-4009-b5ba-99a22d130555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:59.082864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:59.119416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09170adb-d3fc-4009-b5ba-99a22d130555 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:52:59.229180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e87d36-7edf-403c-9e01-f9bc491962fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:52:59.229789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e87d36-7edf-403c-9e01-f9bc491962fe service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:52:59.230066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e87d36-7edf-403c-9e01-f9bc491962fe service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:52:59.230335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e87d36-7edf-403c-9e01-f9bc491962fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:52:59.231972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-30e87d36-7edf-403c-9e01-f9bc491962fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:52:59.310751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:52:59.322996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e87d36-7edf-403c-9e01-f9bc491962fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:52:59.324597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e87d36-7edf-403c-9e01-f9bc491962fe service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:52:59.342775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 709/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:00.458504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f032a-85e5-4105-8da4-81c80f718e12 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:00.459103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f032a-85e5-4105-8da4-81c80f718e12 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:00.459225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f032a-85e5-4105-8da4-81c80f718e12 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:00.460502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:00.466720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6868c6-007b-48e8-a336-242258a7048b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:00.467176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6868c6-007b-48e8-a336-242258a7048b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:00.467290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6868c6-007b-48e8-a336-242258a7048b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:00.543256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cf6868c6-007b-48e8-a336-242258a7048b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:00.544788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cf6868c6-007b-48e8-a336-242258a7048b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:00.640786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:00.791400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f032a-85e5-4105-8da4-81c80f718e12 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:00.792910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-032f032a-85e5-4105-8da4-81c80f718e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:00.877475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:00.939463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f032a-85e5-4105-8da4-81c80f718e12 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:01.048394 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8058382-087d-4584-a677-4f4d299f319c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:01.048992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8058382-087d-4584-a677-4f4d299f319c service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:01.049272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8058382-087d-4584-a677-4f4d299f319c service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:01.049538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8058382-087d-4584-a677-4f4d299f319c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:01.051141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a8058382-087d-4584-a677-4f4d299f319c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:01.098484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:01.140351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8058382-087d-4584-a677-4f4d299f319c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:01.141863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8058382-087d-4584-a677-4f4d299f319c service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:01.159305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 718/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:02.216209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151c38-ebbd-49e1-a3a4-035fb3aa7711 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:02.216648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151c38-ebbd-49e1-a3a4-035fb3aa7711 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:02.216758 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151c38-ebbd-49e1-a3a4-035fb3aa7711 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:02.218087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:02.224484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2803c-d06f-44ee-9feb-acaf6d586926 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:02.224906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2803c-d06f-44ee-9feb-acaf6d586926 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:02.225072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2803c-d06f-44ee-9feb-acaf6d586926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:02.300200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-25b2803c-d06f-44ee-9feb-acaf6d586926 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:02.301407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-25b2803c-d06f-44ee-9feb-acaf6d586926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:02.396651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:02.547671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8151c38-ebbd-49e1-a3a4-035fb3aa7711 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:02.549147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d8151c38-ebbd-49e1-a3a4-035fb3aa7711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:02.597727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:02.694055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8151c38-ebbd-49e1-a3a4-035fb3aa7711 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:02.806741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6cfe1-61a1-43b5-8f2b-5830607085a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:02.807284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6cfe1-61a1-43b5-8f2b-5830607085a1 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:02.807559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6cfe1-61a1-43b5-8f2b-5830607085a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:02.807824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6cfe1-61a1-43b5-8f2b-5830607085a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:02.809404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a4d6cfe1-61a1-43b5-8f2b-5830607085a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:02.816166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:02.897835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6cfe1-61a1-43b5-8f2b-5830607085a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:02.899490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6cfe1-61a1-43b5-8f2b-5830607085a1 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:02.917511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 712/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:03.966402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54ebd8-aee8-4d70-ad56-b94fab8b7283 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:03.966929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54ebd8-aee8-4d70-ad56-b94fab8b7283 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:03.966929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54ebd8-aee8-4d70-ad56-b94fab8b7283 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:03.968246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:03.974262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81bfbd-1f45-46aa-bbab-6d00f30b81b2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:03.974671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81bfbd-1f45-46aa-bbab-6d00f30b81b2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:03.974808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81bfbd-1f45-46aa-bbab-6d00f30b81b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:04.048988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0d81bfbd-1f45-46aa-bbab-6d00f30b81b2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:04.050190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0d81bfbd-1f45-46aa-bbab-6d00f30b81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:04.116339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:04.145587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:04.294293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54ebd8-aee8-4d70-ad56-b94fab8b7283 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:04.295775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ad54ebd8-aee8-4d70-ad56-b94fab8b7283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:04.316233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:04.442123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54ebd8-aee8-4d70-ad56-b94fab8b7283 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:04.526432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:04.551934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc6c69-99fb-414f-accd-b1ebcd3e6df4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:04.552471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6c69-99fb-414f-accd-b1ebcd3e6df4 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:04.552748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6c69-99fb-414f-accd-b1ebcd3e6df4 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:04.553018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6c69-99fb-414f-accd-b1ebcd3e6df4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:04.554591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-80cc6c69-99fb-414f-accd-b1ebcd3e6df4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:04.641559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc6c69-99fb-414f-accd-b1ebcd3e6df4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:04.643189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc6c69-99fb-414f-accd-b1ebcd3e6df4 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:04.661023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 721/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:05.714853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e4d70-f28e-4686-a309-405a6c92dd64 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:05.715417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e4d70-f28e-4686-a309-405a6c92dd64 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:05.715417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e4d70-f28e-4686-a309-405a6c92dd64 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:05.716750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:05.722819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c7fd9-fe92-495b-8987-ccf204461b13 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:05.723244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c7fd9-fe92-495b-8987-ccf204461b13 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:05.723406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c7fd9-fe92-495b-8987-ccf204461b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:05.799028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-217c7fd9-fe92-495b-8987-ccf204461b13 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:05.800232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-217c7fd9-fe92-495b-8987-ccf204461b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:05.841165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:05.896683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:06.047079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e4d70-f28e-4686-a309-405a6c92dd64 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:06.048521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-489e4d70-f28e-4686-a309-405a6c92dd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:06.193692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e4d70-f28e-4686-a309-405a6c92dd64 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:06.248284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:06.306943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59b8df-2c3a-4c12-ae7c-f5d91106cd89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:06.307507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59b8df-2c3a-4c12-ae7c-f5d91106cd89 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:06.307786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59b8df-2c3a-4c12-ae7c-f5d91106cd89 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:06.308053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59b8df-2c3a-4c12-ae7c-f5d91106cd89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:06.309635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4c59b8df-2c3a-4c12-ae7c-f5d91106cd89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:06.401698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59b8df-2c3a-4c12-ae7c-f5d91106cd89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:06.403330 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59b8df-2c3a-4c12-ae7c-f5d91106cd89 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:06.421861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 715/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:07.475918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050dae9-ec8f-46d2-a50d-22a763df4f57 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:07.476396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050dae9-ec8f-46d2-a50d-22a763df4f57 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:07.476491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050dae9-ec8f-46d2-a50d-22a763df4f57 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:07.477816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:07.484014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ea51c-6f49-4635-84b6-b53e1cff5ae1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:07.484396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ea51c-6f49-4635-84b6-b53e1cff5ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:07.484573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ea51c-6f49-4635-84b6-b53e1cff5ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:07.560058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2f2ea51c-6f49-4635-84b6-b53e1cff5ae1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:07.561251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2f2ea51c-6f49-4635-84b6-b53e1cff5ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:07.563277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:07.657353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:07.807705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050dae9-ec8f-46d2-a50d-22a763df4f57 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:07.809201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4050dae9-ec8f-46d2-a50d-22a763df4f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:07.956089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050dae9-ec8f-46d2-a50d-22a763df4f57 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:07.982569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:08.067270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b3a21-739f-4105-ba35-c3f16561d370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:08.067865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b3a21-739f-4105-ba35-c3f16561d370 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:08.068144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b3a21-739f-4105-ba35-c3f16561d370 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:08.068407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b3a21-739f-4105-ba35-c3f16561d370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:08.069974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d03b3a21-739f-4105-ba35-c3f16561d370 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:08.158026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b3a21-739f-4105-ba35-c3f16561d370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:08.159676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b3a21-739f-4105-ba35-c3f16561d370 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:08.177580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 724/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:10.245461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a12e6-375d-4383-9b6e-2620682c6e9b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:10.246022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a12e6-375d-4383-9b6e-2620682c6e9b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:10.246022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a12e6-375d-4383-9b6e-2620682c6e9b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:10.247395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:10.252725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c11fc-083e-4872-bf63-b4fd03657b93 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:10.253140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c11fc-083e-4872-bf63-b4fd03657b93 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:10.253275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c11fc-083e-4872-bf63-b4fd03657b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:10.293113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:10.319685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e31c11fc-083e-4872-bf63-b4fd03657b93 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:10.320877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e31c11fc-083e-4872-bf63-b4fd03657b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:10.414575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:10.699708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a12e6-375d-4383-9b6e-2620682c6e9b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:10.701271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5c3a12e6-375d-4383-9b6e-2620682c6e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:10.803428 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a12e6-375d-4383-9b6e-2620682c6e9b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:10.837676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:10.881278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821d2a-2cf7-4852-b9ea-bd4cbe958c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:10.881880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821d2a-2cf7-4852-b9ea-bd4cbe958c5b service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:10.882159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821d2a-2cf7-4852-b9ea-bd4cbe958c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:10.882764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821d2a-2cf7-4852-b9ea-bd4cbe958c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:10.884976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b821d2a-2cf7-4852-b9ea-bd4cbe958c5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:10.992677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b821d2a-2cf7-4852-b9ea-bd4cbe958c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:10.994661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b821d2a-2cf7-4852-b9ea-bd4cbe958c5b service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:11.012903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 718/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:13.329748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4ca58-6d5f-48a1-8c86-00b7749a4267 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:13.330357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4ca58-6d5f-48a1-8c86-00b7749a4267 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:13.330357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4ca58-6d5f-48a1-8c86-00b7749a4267 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:13.331688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:13.337483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48020250-247d-48a9-966b-f1b93b1c49a2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:13.337935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48020250-247d-48a9-966b-f1b93b1c49a2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:13.338042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48020250-247d-48a9-966b-f1b93b1c49a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:13.403815 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-48020250-247d-48a9-966b-f1b93b1c49a2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:13.405027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-48020250-247d-48a9-966b-f1b93b1c49a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:13.418608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:13.502152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:13.790536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4ca58-6d5f-48a1-8c86-00b7749a4267 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:13.792115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-06b4ca58-6d5f-48a1-8c86-00b7749a4267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:13.893352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4ca58-6d5f-48a1-8c86-00b7749a4267 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:13.906543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:13.967321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad983dd-3d12-4af6-b41d-e146c5401a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:13.967838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad983dd-3d12-4af6-b41d-e146c5401a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:13.968112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad983dd-3d12-4af6-b41d-e146c5401a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:13.968373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad983dd-3d12-4af6-b41d-e146c5401a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:13.969944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8ad983dd-3d12-4af6-b41d-e146c5401a2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:14.076233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad983dd-3d12-4af6-b41d-e146c5401a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:14.077681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad983dd-3d12-4af6-b41d-e146c5401a2b service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:14.095701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 727/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:16.516423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b45a8-11c8-469c-aa3d-7a80b366ed3a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:16.516946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b45a8-11c8-469c-aa3d-7a80b366ed3a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:16.516946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b45a8-11c8-469c-aa3d-7a80b366ed3a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:16.518302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:53:16.523626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e920ae-c280-488e-b330-5b8c7d285079 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:16.524077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e920ae-c280-488e-b330-5b8c7d285079 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:16.524700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e920ae-c280-488e-b330-5b8c7d285079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:16.572982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:16.591757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-04e920ae-c280-488e-b330-5b8c7d285079 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:16.592955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-04e920ae-c280-488e-b330-5b8c7d285079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:16.686992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:16.966967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b45a8-11c8-469c-aa3d-7a80b366ed3a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:16.968550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-951b45a8-11c8-469c-aa3d-7a80b366ed3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:17.070048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b45a8-11c8-469c-aa3d-7a80b366ed3a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:17.110015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:17.145089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a5256-3bf5-460d-b125-95a36c690dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:17.145695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a5256-3bf5-460d-b125-95a36c690dca service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:17.145981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a5256-3bf5-460d-b125-95a36c690dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:17.146254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a5256-3bf5-460d-b125-95a36c690dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:17.147887 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a09a5256-3bf5-460d-b125-95a36c690dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:17.376810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a5256-3bf5-460d-b125-95a36c690dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:17.378239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a5256-3bf5-460d-b125-95a36c690dca service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:17.395854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 721/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:19.634994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449bda4-edf3-42a6-aea3-a407af340b27 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:19.635511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449bda4-edf3-42a6-aea3-a407af340b27 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:19.635511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449bda4-edf3-42a6-aea3-a407af340b27 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:19.636817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:19.642231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81b72-2bd0-4de0-8bf5-113f5b3c263c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:19.642711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81b72-2bd0-4de0-8bf5-113f5b3c263c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:19.642820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81b72-2bd0-4de0-8bf5-113f5b3c263c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:19.707382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b5f81b72-2bd0-4de0-8bf5-113f5b3c263c None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:19.708570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b5f81b72-2bd0-4de0-8bf5-113f5b3c263c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:19.728924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:19.804102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:20.087713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449bda4-edf3-42a6-aea3-a407af340b27 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:20.089230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c449bda4-edf3-42a6-aea3-a407af340b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:20.189580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449bda4-edf3-42a6-aea3-a407af340b27 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:20.209211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:20.262689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c51435-da21-46a1-be2d-9776e7ffe0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:20.263321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51435-da21-46a1-be2d-9776e7ffe0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:20.263596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51435-da21-46a1-be2d-9776e7ffe0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:20.263864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51435-da21-46a1-be2d-9776e7ffe0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:20.265441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f3c51435-da21-46a1-be2d-9776e7ffe0e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:20.371185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c51435-da21-46a1-be2d-9776e7ffe0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:20.372846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c51435-da21-46a1-be2d-9776e7ffe0e9 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:20.390073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 730/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:22.578192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77bf9-72fb-47a3-9616-0d927d5d316d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:22.578772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77bf9-72fb-47a3-9616-0d927d5d316d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:22.578772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77bf9-72fb-47a3-9616-0d927d5d316d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:22.580109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:22.585390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320a959-e40d-43b1-ac5d-e912941c3839 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:22.585787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320a959-e40d-43b1-ac5d-e912941c3839 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:22.585931 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320a959-e40d-43b1-ac5d-e912941c3839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:22.648191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:22.651444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3320a959-e40d-43b1-ac5d-e912941c3839 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:22.652891 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3320a959-e40d-43b1-ac5d-e912941c3839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:22.748635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:23.030760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77bf9-72fb-47a3-9616-0d927d5d316d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:23.032314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-95d77bf9-72fb-47a3-9616-0d927d5d316d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:23.134653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77bf9-72fb-47a3-9616-0d927d5d316d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:23.184140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:23.208748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e32dc4-f933-4b92-8c68-1f85a9675496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:23.209345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e32dc4-f933-4b92-8c68-1f85a9675496 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:23.209620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e32dc4-f933-4b92-8c68-1f85a9675496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:23.209884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e32dc4-f933-4b92-8c68-1f85a9675496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:23.211501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b8e32dc4-f933-4b92-8c68-1f85a9675496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:23.317678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e32dc4-f933-4b92-8c68-1f85a9675496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:23.319189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e32dc4-f933-4b92-8c68-1f85a9675496 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:23.337160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 724/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:25.520374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c9337-cadb-4e1f-ab44-7b078d9e6b35 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:25.520902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c9337-cadb-4e1f-ab44-7b078d9e6b35 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:25.520902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c9337-cadb-4e1f-ab44-7b078d9e6b35 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:25.522241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:25.528498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f5cb2-1ef1-4018-b8f6-e6f1e61f6cc2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:25.530786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f5cb2-1ef1-4018-b8f6-e6f1e61f6cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:25.530786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f5cb2-1ef1-4018-b8f6-e6f1e61f6cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:25.597836 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-127f5cb2-1ef1-4018-b8f6-e6f1e61f6cc2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:25.599065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-127f5cb2-1ef1-4018-b8f6-e6f1e61f6cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:25.628427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:25.694802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:25.975683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c9337-cadb-4e1f-ab44-7b078d9e6b35 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:25.977197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4d5c9337-cadb-4e1f-ab44-7b078d9e6b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:26.080114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c9337-cadb-4e1f-ab44-7b078d9e6b35 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:26.111352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:26.156502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4efbc8-34b6-48a2-937b-38cdf22ee4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:26.157098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4efbc8-34b6-48a2-937b-38cdf22ee4bb service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:26.157375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4efbc8-34b6-48a2-937b-38cdf22ee4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:26.157640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4efbc8-34b6-48a2-937b-38cdf22ee4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:26.159259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1f4efbc8-34b6-48a2-937b-38cdf22ee4bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:26.264471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4efbc8-34b6-48a2-937b-38cdf22ee4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:26.265815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4efbc8-34b6-48a2-937b-38cdf22ee4bb service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:26.283758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 733/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:53:28.510197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd0931-44d6-45be-be30-e9da4c36fbe2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:28.510786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd0931-44d6-45be-be30-e9da4c36fbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:28.510786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd0931-44d6-45be-be30-e9da4c36fbe2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:28.512099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:28.517258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b013e46-a724-4151-ad47-5d2564515cfc None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:28.517658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b013e46-a724-4151-ad47-5d2564515cfc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:28.517815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b013e46-a724-4151-ad47-5d2564515cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:28.583100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5b013e46-a724-4151-ad47-5d2564515cfc None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:28.584305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5b013e46-a724-4151-ad47-5d2564515cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:28.592422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:28.684313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:28.964694 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd0931-44d6-45be-be30-e9da4c36fbe2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:28.966226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-53cd0931-44d6-45be-be30-e9da4c36fbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:29.069487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd0931-44d6-45be-be30-e9da4c36fbe2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:29.126390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:29.145252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d21057-a22a-4571-ac74-bbbc532f59d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:29.145853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d21057-a22a-4571-ac74-bbbc532f59d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:29.146136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d21057-a22a-4571-ac74-bbbc532f59d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:29.146405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d21057-a22a-4571-ac74-bbbc532f59d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:29.148050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a2d21057-a22a-4571-ac74-bbbc532f59d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:29.255190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d21057-a22a-4571-ac74-bbbc532f59d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:29.256540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d21057-a22a-4571-ac74-bbbc532f59d3 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:29.274476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 727/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:31.468080 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21144a6-0fd4-4fda-8504-1853de3f5343 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:31.468633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21144a6-0fd4-4fda-8504-1853de3f5343 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:31.468633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21144a6-0fd4-4fda-8504-1853de3f5343 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:31.469945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:31.475172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f58e-f71d-4fac-ab8d-7032e5f30eb3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:31.475588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f58e-f71d-4fac-ab8d-7032e5f30eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:31.475717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f58e-f71d-4fac-ab8d-7032e5f30eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:31.539628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9840f58e-f71d-4fac-ab8d-7032e5f30eb3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:31.540875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9840f58e-f71d-4fac-ab8d-7032e5f30eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:31.579266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:31.636865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:31.917121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21144a6-0fd4-4fda-8504-1853de3f5343 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:31.918704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d21144a6-0fd4-4fda-8504-1853de3f5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:32.022374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21144a6-0fd4-4fda-8504-1853de3f5343 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:32.059354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:32.094822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbea88-0c28-4ec2-85ba-2c6cff9ee5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:32.095448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbea88-0c28-4ec2-85ba-2c6cff9ee5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:32.095729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbea88-0c28-4ec2-85ba-2c6cff9ee5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:32.095995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbea88-0c28-4ec2-85ba-2c6cff9ee5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:32.097581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a8bbea88-0c28-4ec2-85ba-2c6cff9ee5b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:32.206519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbea88-0c28-4ec2-85ba-2c6cff9ee5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:32.207974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbea88-0c28-4ec2-85ba-2c6cff9ee5b1 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:32.225371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 736/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:34.436636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eefdaa-9ccc-4042-a0e0-55764803705d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:34.437096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eefdaa-9ccc-4042-a0e0-55764803705d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:34.437205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eefdaa-9ccc-4042-a0e0-55764803705d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:34.438529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:34.443867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585def3-75ec-453b-ac25-58bd87a6a8ed None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:34.444290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585def3-75ec-453b-ac25-58bd87a6a8ed None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:34.444419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585def3-75ec-453b-ac25-58bd87a6a8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:34.510041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d585def3-75ec-453b-ac25-58bd87a6a8ed None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:34.511277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d585def3-75ec-453b-ac25-58bd87a6a8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:34.529456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:34.608254 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:34.888109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eefdaa-9ccc-4042-a0e0-55764803705d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:34.889641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c1eefdaa-9ccc-4042-a0e0-55764803705d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:34.992894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eefdaa-9ccc-4042-a0e0-55764803705d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:35.062382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:35.070049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04090273-7aa7-47d1-bc8e-17986e58d9e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:35.070653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090273-7aa7-47d1-bc8e-17986e58d9e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:35.071106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090273-7aa7-47d1-bc8e-17986e58d9e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:35.071379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090273-7aa7-47d1-bc8e-17986e58d9e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:35.072967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-04090273-7aa7-47d1-bc8e-17986e58d9e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:35.188079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090273-7aa7-47d1-bc8e-17986e58d9e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:35.189422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090273-7aa7-47d1-bc8e-17986e58d9e0 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:35.207989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 730/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:37.359739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8987b-4f20-4033-917c-a9be0598a0e6 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:37.360303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8987b-4f20-4033-917c-a9be0598a0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:37.360303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8987b-4f20-4033-917c-a9be0598a0e6 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:37.361614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:37.367055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36190544-905a-485f-9aec-7ade21973c21 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:37.367452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36190544-905a-485f-9aec-7ade21973c21 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:37.367628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36190544-905a-485f-9aec-7ade21973c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:37.433716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-36190544-905a-485f-9aec-7ade21973c21 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:37.434960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36190544-905a-485f-9aec-7ade21973c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:37.482177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:37.531125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:37.814000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8987b-4f20-4033-917c-a9be0598a0e6 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:37.815623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fea8987b-4f20-4033-917c-a9be0598a0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:37.917357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8987b-4f20-4033-917c-a9be0598a0e6 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:37.964200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:37.989737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd2cbb-70f0-46e0-bf63-a4514f5d5fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:37.990342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd2cbb-70f0-46e0-bf63-a4514f5d5fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:37.990620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd2cbb-70f0-46e0-bf63-a4514f5d5fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:37.990885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd2cbb-70f0-46e0-bf63-a4514f5d5fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:37.992493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e9dd2cbb-70f0-46e0-bf63-a4514f5d5fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:38.097710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd2cbb-70f0-46e0-bf63-a4514f5d5fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:38.099179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd2cbb-70f0-46e0-bf63-a4514f5d5fb6 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:38.116973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 739/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:53:47.529539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=22 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:48.114747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=66 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:57.892209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724323-33d5-4d07-b677-5f7838121a8b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:57.892820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724323-33d5-4d07-b677-5f7838121a8b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:57.892820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724323-33d5-4d07-b677-5f7838121a8b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:57.894094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 732/1471] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:57.899983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51d89d-ad1f-4ef9-92fb-085944cb7721 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:57.900426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51d89d-ad1f-4ef9-92fb-085944cb7721 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:57.900522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51d89d-ad1f-4ef9-92fb-085944cb7721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:57.965002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5a51d89d-ad1f-4ef9-92fb-085944cb7721 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:57.966187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5a51d89d-ad1f-4ef9-92fb-085944cb7721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:58.060518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 740/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:58.362999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19462e9f-1ba7-4f98-a321-2b874108b994 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:58.363293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19462e9f-1ba7-4f98-a321-2b874108b994 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:58.363566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19462e9f-1ba7-4f98-a321-2b874108b994 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:58.364884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 733/1473] 10.222.0.52 () {54 vars in 962 bytes} [Mon Oct 10 11:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:53:58.369870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c61ae-ea00-4d83-9a2e-d6a41b372243 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:58.370157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c61ae-ea00-4d83-9a2e-d6a41b372243 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:58.370544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c61ae-ea00-4d83-9a2e-d6a41b372243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:58.445512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e07c61ae-ea00-4d83-9a2e-d6a41b372243 None None] MFA Rules not processed for user `c6774fdd8b2e46b6a7d00919568f9fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:53:58.446726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e07c61ae-ea00-4d83-9a2e-d6a41b372243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:58.466828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:58.519737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 741/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Mon Oct 10 11:53:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:53:58.527824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19462e9f-1ba7-4f98-a321-2b874108b994 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:53:58.529360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-19462e9f-1ba7-4f98-a321-2b874108b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:58.632314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19462e9f-1ba7-4f98-a321-2b874108b994 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:53:58.708666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec568c3d-03db-4666-b5cc-3604f6be502b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6774fdd8b2e46b6a7d00919568f9fd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:53:58.709265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568c3d-03db-4666-b5cc-3604f6be502b service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:53:58.709537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568c3d-03db-4666-b5cc-3604f6be502b service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:53:58.709799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec568c3d-03db-4666-b5cc-3604f6be502b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:53:58.711416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ec568c3d-03db-4666-b5cc-3604f6be502b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:53:58.764427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:53:58.819140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec568c3d-03db-4666-b5cc-3604f6be502b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:53:58.820587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec568c3d-03db-4666-b5cc-3604f6be502b service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:53:58.839458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 734/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:54:01.060352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36784c-6f05-411e-b71b-2a09b9a81384 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:01.060915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36784c-6f05-411e-b71b-2a09b9a81384 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:01.060915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36784c-6f05-411e-b71b-2a09b9a81384 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:01.062267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 742/1476] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:54:01.068367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2a16-5442-430d-a424-3cf744ac30b1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:01.068793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2a16-5442-430d-a424-3cf744ac30b1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:01.068903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2a16-5442-430d-a424-3cf744ac30b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:01.133650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e8fc2a16-5442-430d-a424-3cf744ac30b1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:54:01.134949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e8fc2a16-5442-430d-a424-3cf744ac30b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:01.229266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 735/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:54:01.502097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36784c-6f05-411e-b71b-2a09b9a81384 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:54:01.503720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ea36784c-6f05-411e-b71b-2a09b9a81384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:01.627107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36784c-6f05-411e-b71b-2a09b9a81384 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:54:01.674479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:01.719507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141de756-e573-460f-94fe-f16601490dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:54:01.720110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141de756-e573-460f-94fe-f16601490dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:01.720392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141de756-e573-460f-94fe-f16601490dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:01.720663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141de756-e573-460f-94fe-f16601490dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:01.721290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141de756-e573-460f-94fe-f16601490dfa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:54:01.723043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING py.warnings [None req-141de756-e573-460f-94fe-f16601490dfa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:54:01.723043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: warnings.warn(msg) Oct 10 11:54:01.723043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: Oct 10 11:54:01.724170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141de756-e573-460f-94fe-f16601490dfa admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:54:01.742574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 743/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Mon Oct 10 11:54:01 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:54:03.918546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4979e4-c7cc-4231-bb1c-22db222f5234 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:03.919150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4979e4-c7cc-4231-bb1c-22db222f5234 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:03.919150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4979e4-c7cc-4231-bb1c-22db222f5234 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:03.920475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 736/1479] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:54:03.925877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32531b-2a26-4180-a1da-814d8901ce83 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:03.926311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32531b-2a26-4180-a1da-814d8901ce83 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:03.926423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32531b-2a26-4180-a1da-814d8901ce83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:03.990785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-4d32531b-2a26-4180-a1da-814d8901ce83 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:54:03.991930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4d32531b-2a26-4180-a1da-814d8901ce83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:04.088543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 744/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:54:04.385769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4979e4-c7cc-4231-bb1c-22db222f5234 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:54:04.387328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0a4979e4-c7cc-4231-bb1c-22db222f5234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:04.396903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:04.531557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4979e4-c7cc-4231-bb1c-22db222f5234 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:54:04.609147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:04.643226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99bb04-b909-4747-98e8-c6fbd2c5527e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:54:04.643747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99bb04-b909-4747-98e8-c6fbd2c5527e service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:04.644022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99bb04-b909-4747-98e8-c6fbd2c5527e service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:04.644286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99bb04-b909-4747-98e8-c6fbd2c5527e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:04.645865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cc99bb04-b909-4747-98e8-c6fbd2c5527e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:04.735032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99bb04-b909-4747-98e8-c6fbd2c5527e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:54:04.736634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99bb04-b909-4747-98e8-c6fbd2c5527e service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:54:04.754483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 737/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:54:07.084596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d919-8223-4a79-83fe-56505d5f0b4e None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:07.085164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d919-8223-4a79-83fe-56505d5f0b4e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:07.085164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d919-8223-4a79-83fe-56505d5f0b4e None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:07.086607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 745/1482] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:54:07.092454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765b057-b346-4aab-9ad2-4e57f54deb34 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:07.092862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765b057-b346-4aab-9ad2-4e57f54deb34 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:07.093001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765b057-b346-4aab-9ad2-4e57f54deb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:07.159052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9765b057-b346-4aab-9ad2-4e57f54deb34 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:54:07.160263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9765b057-b346-4aab-9ad2-4e57f54deb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:07.206449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:07.256230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 738/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:54:07.541866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223d919-8223-4a79-83fe-56505d5f0b4e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:54:07.543425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c223d919-8223-4a79-83fe-56505d5f0b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:07.567430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:07.644872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223d919-8223-4a79-83fe-56505d5f0b4e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:54:07.718714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dd3d2-8360-40a0-80a3-df2d8b87ac1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:54:07.719272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd3d2-8360-40a0-80a3-df2d8b87ac1f service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:07.719582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd3d2-8360-40a0-80a3-df2d8b87ac1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:07.719857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd3d2-8360-40a0-80a3-df2d8b87ac1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:07.721456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-be1dd3d2-8360-40a0-80a3-df2d8b87ac1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:07.794109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:07.828894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dd3d2-8360-40a0-80a3-df2d8b87ac1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:54:07.830323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dd3d2-8360-40a0-80a3-df2d8b87ac1f service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:54:07.848448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 746/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:54:10.037450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434baded-4719-424e-9a55-09e28ea2037e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:10.038041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434baded-4719-424e-9a55-09e28ea2037e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:10.038041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434baded-4719-424e-9a55-09e28ea2037e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:10.039495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 739/1485] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:54:10.044672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95622-845e-4fa6-b7da-c98c562a7a2a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:10.045119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95622-845e-4fa6-b7da-c98c562a7a2a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:10.045252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95622-845e-4fa6-b7da-c98c562a7a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:10.110853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-0fc95622-845e-4fa6-b7da-c98c562a7a2a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:54:10.112090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0fc95622-845e-4fa6-b7da-c98c562a7a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:10.208652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 747/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:54:10.503899 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434baded-4719-424e-9a55-09e28ea2037e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:54:10.505438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-434baded-4719-424e-9a55-09e28ea2037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:10.610572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434baded-4719-424e-9a55-09e28ea2037e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:54:10.682490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:10.683933 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b4cce-a74e-4f8e-9014-a860bc3f1d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:54:10.684532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b4cce-a74e-4f8e-9014-a860bc3f1d4e service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:10.685001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b4cce-a74e-4f8e-9014-a860bc3f1d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:10.685429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b4cce-a74e-4f8e-9014-a860bc3f1d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:10.687221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-257b4cce-a74e-4f8e-9014-a860bc3f1d4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:10.797043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b4cce-a74e-4f8e-9014-a860bc3f1d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:54:10.798469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b4cce-a74e-4f8e-9014-a860bc3f1d4e service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:54:10.816807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 740/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:54:13.066961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8e9d5-e7cb-482e-8b05-14c7de507aa3 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:13.067528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8e9d5-e7cb-482e-8b05-14c7de507aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:13.067528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8e9d5-e7cb-482e-8b05-14c7de507aa3 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:13.068946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 748/1488] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:54:13.074671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046a22-9e5d-496c-8793-fbad4a4f25de None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:13.075087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046a22-9e5d-496c-8793-fbad4a4f25de None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:13.075255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046a22-9e5d-496c-8793-fbad4a4f25de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:13.141148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7d046a22-9e5d-496c-8793-fbad4a4f25de None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:54:13.142358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7d046a22-9e5d-496c-8793-fbad4a4f25de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:13.195199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:13.239031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 741/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:54:13.535087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8e9d5-e7cb-482e-8b05-14c7de507aa3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:54:13.536643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d9f8e9d5-e7cb-482e-8b05-14c7de507aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:13.573330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:13.638585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8e9d5-e7cb-482e-8b05-14c7de507aa3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:54:13.711734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dfabb-8474-450f-95a0-6b73944c7010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:54:13.712958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dfabb-8474-450f-95a0-6b73944c7010 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:13.713242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dfabb-8474-450f-95a0-6b73944c7010 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:13.713515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dfabb-8474-450f-95a0-6b73944c7010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:13.715129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9a1dfabb-8474-450f-95a0-6b73944c7010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:13.793208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:13.822199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dfabb-8474-450f-95a0-6b73944c7010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:54:13.823660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dfabb-8474-450f-95a0-6b73944c7010 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:54:13.841039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 749/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:54:16.192851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f143bf-ebaa-4d6a-871a-17de4681ea12 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:16.193467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f143bf-ebaa-4d6a-871a-17de4681ea12 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:16.193467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f143bf-ebaa-4d6a-871a-17de4681ea12 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:16.194746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 742/1491] 10.222.0.52 () {54 vars in 927 bytes} [Mon Oct 10 11:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:54:16.200017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a0288-4f75-487e-bf09-71b013439894 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:16.200534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a0288-4f75-487e-bf09-71b013439894 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:16.200634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a0288-4f75-487e-bf09-71b013439894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:16.265010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f79a0288-4f75-487e-bf09-71b013439894 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:54:16.266208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f79a0288-4f75-487e-bf09-71b013439894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:16.364383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 750/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Mon Oct 10 11:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:54:16.655403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f143bf-ebaa-4d6a-871a-17de4681ea12 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:54:16.656940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-43f143bf-ebaa-4d6a-871a-17de4681ea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:16.758863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f143bf-ebaa-4d6a-871a-17de4681ea12 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:54:16.833118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844fe0c-b2c3-421e-963a-92aaa7ef5cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:54:16.833715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844fe0c-b2c3-421e-963a-92aaa7ef5cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:54:16.833991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844fe0c-b2c3-421e-963a-92aaa7ef5cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:54:16.834255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844fe0c-b2c3-421e-963a-92aaa7ef5cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:54:16.835868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e844fe0c-b2c3-421e-963a-92aaa7ef5cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:54:16.844270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:16.942684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844fe0c-b2c3-421e-963a-92aaa7ef5cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:54:16.944056 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844fe0c-b2c3-421e-963a-92aaa7ef5cf9 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:54:16.961710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 743/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:54:26.362848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=82 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:54:26.959432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=50 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:35.036796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0f35c-b72c-4ee3-8cbf-e4cd8f6ea2a1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:35.037398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0f35c-b72c-4ee3-8cbf-e4cd8f6ea2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:35.037398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0f35c-b72c-4ee3-8cbf-e4cd8f6ea2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:35.112687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-21d0f35c-b72c-4ee3-8cbf-e4cd8f6ea2a1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:55:35.113903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-21d0f35c-b72c-4ee3-8cbf-e4cd8f6ea2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:35.210589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 751/1494] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:55:35.218821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbbc5f-94a1-4ef0-99e7-ffc7c3108bb5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:35.219349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbbc5f-94a1-4ef0-99e7-ffc7c3108bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:35.219450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbbc5f-94a1-4ef0-99e7-ffc7c3108bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:35.295130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-20bbbc5f-94a1-4ef0-99e7-ffc7c3108bb5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:55:35.296248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-20bbbc5f-94a1-4ef0-99e7-ffc7c3108bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:35.391424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 744/1495] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:55:35.396010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0f35c-b72c-4ee3-8cbf-e4cd8f6ea2a1 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:35.397322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-21d0f35c-b72c-4ee3-8cbf-e4cd8f6ea2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:35.409321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0f35c-b72c-4ee3-8cbf-e4cd8f6ea2a1 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:35.470486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:35.507462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:35.508076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:35.508422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:35.508695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:35.509333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:55:35.509631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:35.510653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING py.warnings [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:55:35.510653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: warnings.warn(msg) Oct 10 11:55:35.510653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: Oct 10 11:55:35.511725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:35.523461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 752/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:55:35.531904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbbc5f-94a1-4ef0-99e7-ffc7c3108bb5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:35.542004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbbc5f-94a1-4ef0-99e7-ffc7c3108bb5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:35.611734 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:35.645561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:35.646160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:35.646433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:35.646697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:35.647342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:35.649059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:55:35.649059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Oct 10 11:55:35.649059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: Oct 10 11:55:35.650786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:35.707784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 745/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:55:35 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:55:35.711630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:35.713760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:35.748725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b28e55-9765-4cdf-a6f6-40d8aa12ff16 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:35.841122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:35.841737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:35.841996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:35.842263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:35.842914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:35.843919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING py.warnings [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:55:35.843919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: warnings.warn(msg) Oct 10 11:55:35.843919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: Oct 10 11:55:35.845121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:35.886572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 753/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:55:35 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:55:35.890716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:35.892326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:35.953308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901075b-ca44-4098-a554-5ea7c507510c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:36.035992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:36.049071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:36.049747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:36.050026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:36.050293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:36.050991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:36.051996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:55:36.051996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Oct 10 11:55:36.051996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: Oct 10 11:55:36.052873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:36.065321 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 746/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:55:36.069438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:36.070955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:36.082541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8cfcf-2b9c-438b-807b-3b0ee9456eed admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:36.107142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:36.177131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb65cd1-8b34-4a07-960a-888c1699b885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:36.177639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb65cd1-8b34-4a07-960a-888c1699b885 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:36.177914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb65cd1-8b34-4a07-960a-888c1699b885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:36.178180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb65cd1-8b34-4a07-960a-888c1699b885 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:36.178809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb65cd1-8b34-4a07-960a-888c1699b885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:36.179846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING py.warnings [None req-fcb65cd1-8b34-4a07-960a-888c1699b885 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:55:36.179846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: warnings.warn(msg) Oct 10 11:55:36.179846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: Oct 10 11:55:36.180725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb65cd1-8b34-4a07-960a-888c1699b885 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:36.192883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 754/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:55:36.197711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:36.199277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:36.233184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1b950-15c1-4476-aa79-67955e4fc58c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:36.323422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120483e7-7afd-44e8-a4e7-9e10b2203729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:36.324065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120483e7-7afd-44e8-a4e7-9e10b2203729 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:36.324337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120483e7-7afd-44e8-a4e7-9e10b2203729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:36.324601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120483e7-7afd-44e8-a4e7-9e10b2203729 admin admin] PATH_INFO: `/v3/projects/c16091560e1548ac84b7061f66425511/users/ba04563f0f844fe6836c24af00c501f8/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:36.346798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120483e7-7afd-44e8-a4e7-9e10b2203729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16091560e1548ac84b7061f66425511, user_id=ba04563f0f844fe6836c24af00c501f8, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:36.348122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-120483e7-7afd-44e8-a4e7-9e10b2203729 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:55:36.348122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Oct 10 11:55:36.348122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: Oct 10 11:55:36.349762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120483e7-7afd-44e8-a4e7-9e10b2203729 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:36.373506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 747/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:55:36 2022] PUT /identity/v3/projects/c16091560e1548ac84b7061f66425511/users/ba04563f0f844fe6836c24af00c501f8/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:55:36.386062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3d011-4960-4798-9c20-789d26c805be None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:36.386374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3d011-4960-4798-9c20-789d26c805be None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:36.386662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3d011-4960-4798-9c20-789d26c805be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:36.444058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-17c3d011-4960-4798-9c20-789d26c805be None None] MFA Rules not processed for user `ba04563f0f844fe6836c24af00c501f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:55:36.445191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-17c3d011-4960-4798-9c20-789d26c805be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:36.519780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 755/1502] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:55:36.533092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1bb2-0771-41bf-91b3-65becdc9b46e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:36.533362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1bb2-0771-41bf-91b3-65becdc9b46e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:36.533633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1bb2-0771-41bf-91b3-65becdc9b46e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:36.535049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 748/1503] 10.222.0.52 () {54 vars in 963 bytes} [Mon Oct 10 11:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:55:36.539796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2818680-192c-45a6-82d2-a76215af76b0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:36.540088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2818680-192c-45a6-82d2-a76215af76b0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:36.540364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2818680-192c-45a6-82d2-a76215af76b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:36.571400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:36.615738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b2818680-192c-45a6-82d2-a76215af76b0 None None] MFA Rules not processed for user `75aa23cd162e4da2bd6a936c27e51f41`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:55:36.616941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b2818680-192c-45a6-82d2-a76215af76b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:36.691631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 756/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Mon Oct 10 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:55:36.699950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1bb2-0771-41bf-91b3-65becdc9b46e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:36.701399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7fff1bb2-0771-41bf-91b3-65becdc9b46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:36.750374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:36.809903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1bb2-0771-41bf-91b3-65becdc9b46e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:36.882111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:36.882743 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:36.883060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:36.883333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:36.884941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:36.981304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:36.982658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:36.986150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:37.001199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 749/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:55:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:55:37.045709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1655f-356f-4d3d-836e-0b82888a8165 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:37.046000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1655f-356f-4d3d-836e-0b82888a8165 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:37.046276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1655f-356f-4d3d-836e-0b82888a8165 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:37.047582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 757/1506] 10.222.0.52 () {52 vars in 830 bytes} [Mon Oct 10 11:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:55:37.051209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:37.052414 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:37.151617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc0685-eed6-43e0-9f91-a22f354d7b0d service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:37.242938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:37.243555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:37.243763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:37.244030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] PATH_INFO: `/v3/users/ba04563f0f844fe6836c24af00c501f8` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:37.256470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba04563f0f844fe6836c24af00c501f8)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:37.257518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:55:37.257518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Oct 10 11:55:37.257518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: Oct 10 11:55:37.258720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:37.268177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:37.322897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba04563f0f844fe6836c24af00c501f8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:37.336116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba04563f0f844fe6836c24af00c501f8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:37.336564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba04563f0f844fe6836c24af00c501f8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:37.337088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba04563f0f844fe6836c24af00c501f8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:37.347060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba04563f0f844fe6836c24af00c501f8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:37.375550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 750/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:55:37 2022] DELETE /identity/v3/users/ba04563f0f844fe6836c24af00c501f8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:55:37.380004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719538cb-78b1-4801-b349-a9ebc7536f56 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:37.380465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719538cb-78b1-4801-b349-a9ebc7536f56 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:37.380574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719538cb-78b1-4801-b349-a9ebc7536f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:37.439911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-719538cb-78b1-4801-b349-a9ebc7536f56 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:55:37.441119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-719538cb-78b1-4801-b349-a9ebc7536f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:37.516428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:37.529378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 758/1508] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:55:37.538900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:37.540125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:37.637794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7a8d7-d074-4e84-ba74-892b52b80f98 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:37.693477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:37.715299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741b2ea-5c84-4bbf-ac67-6027e54bcec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:37.715913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741b2ea-5c84-4bbf-ac67-6027e54bcec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:37.716193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741b2ea-5c84-4bbf-ac67-6027e54bcec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:37.716489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741b2ea-5c84-4bbf-ac67-6027e54bcec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:37.718077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f741b2ea-5c84-4bbf-ac67-6027e54bcec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:37.826705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741b2ea-5c84-4bbf-ac67-6027e54bcec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:37.828166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741b2ea-5c84-4bbf-ac67-6027e54bcec0 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:37.846058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 751/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:55:38.161826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719538cb-78b1-4801-b349-a9ebc7536f56 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:55:38.163090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-719538cb-78b1-4801-b349-a9ebc7536f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:55:38.174423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719538cb-78b1-4801-b349-a9ebc7536f56 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:55:38.272418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:55:38.273041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:55:38.273317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:55:38.273584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] PATH_INFO: `/v3/projects/c16091560e1548ac84b7061f66425511` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:55:38.283301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c16091560e1548ac84b7061f66425511)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:55:38.284315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING py.warnings [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:55:38.284315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: warnings.warn(msg) Oct 10 11:55:38.284315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: Oct 10 11:55:38.285501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:55:38.342234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c16091560e1548ac84b7061f66425511', 'request_id': 'req-84d1ced6-c1b4-400e-9218-5f1049d49c9b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:38.352115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c16091560e1548ac84b7061f66425511', 'request_id': 'req-84d1ced6-c1b4-400e-9218-5f1049d49c9b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:38.365637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c16091560e1548ac84b7061f66425511', 'request_id': 'req-84d1ced6-c1b4-400e-9218-5f1049d49c9b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:38.387876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] The token cache is being invalidate because project c16091560e1548ac84b7061f66425511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:55:38.388480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:38.388707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-84d1ced6-c1b4-400e-9218-5f1049d49c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:55:38.390063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 759/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:55:38 2022] DELETE /identity/v3/projects/c16091560e1548ac84b7061f66425511 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:55:47.843845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=62 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:55:48.385915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:57:04.515091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410ae41-659e-4c0d-a081-092deb239e58 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:04.515668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410ae41-659e-4c0d-a081-092deb239e58 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:04.515668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410ae41-659e-4c0d-a081-092deb239e58 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:04.517017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:04.521724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ada852-c678-4893-8395-19693ff2a363 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:04.522135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ada852-c678-4893-8395-19693ff2a363 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:04.522265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ada852-c678-4893-8395-19693ff2a363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:04.603759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-12ada852-c678-4893-8395-19693ff2a363 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:04.604966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-12ada852-c678-4893-8395-19693ff2a363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:04.705247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:57:04.859032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410ae41-659e-4c0d-a081-092deb239e58 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:04.861179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9410ae41-659e-4c0d-a081-092deb239e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:04.977927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410ae41-659e-4c0d-a081-092deb239e58 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:04.983844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab6810-e7c3-4660-b03e-9d264f6a11d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:04.984440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6810-e7c3-4660-b03e-9d264f6a11d0 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:04.984723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6810-e7c3-4660-b03e-9d264f6a11d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:04.984997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6810-e7c3-4660-b03e-9d264f6a11d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:04.992974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab6810-e7c3-4660-b03e-9d264f6a11d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:04.994535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab6810-e7c3-4660-b03e-9d264f6a11d0 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:05.003985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 753/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:57:05.584975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f31c2-a668-4bc3-88ae-61de6a420eb0 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:05.585529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f31c2-a668-4bc3-88ae-61de6a420eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:05.585529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f31c2-a668-4bc3-88ae-61de6a420eb0 None None] PATH_INFO: `/v3` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:05.586837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 761/1514] 10.222.0.58 () {54 vars in 925 bytes} [Mon Oct 10 11:57:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:57:05.593602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd51fd-57aa-42e6-a63d-5996ba982d8c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:05.594028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd51fd-57aa-42e6-a63d-5996ba982d8c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:05.594143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd51fd-57aa-42e6-a63d-5996ba982d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:05.653570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-65bd51fd-57aa-42e6-a63d-5996ba982d8c None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:05.654763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-65bd51fd-57aa-42e6-a63d-5996ba982d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:05.722019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:57:05.727642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 754/1515] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Oct 10 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:57:06.086851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf73356-1c97-47ce-979b-e6987829328d None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:06.087193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf73356-1c97-47ce-979b-e6987829328d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:06.087466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf73356-1c97-47ce-979b-e6987829328d None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:06.089433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 762/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:57:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:06.095683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc6ff1-679c-42e9-9803-3f67302575d2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:06.095974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc6ff1-679c-42e9-9803-3f67302575d2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:06.096246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc6ff1-679c-42e9-9803-3f67302575d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:06.115529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9bbc6ff1-679c-42e9-9803-3f67302575d2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:06.116726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9bbc6ff1-679c-42e9-9803-3f67302575d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:06.124429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 755/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:57:06.275642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf73356-1c97-47ce-979b-e6987829328d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:06.290134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf73356-1c97-47ce-979b-e6987829328d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:06.295401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf78e-7afd-4db0-b15a-72ebe9164bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:06.295992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf78e-7afd-4db0-b15a-72ebe9164bb4 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:06.296270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf78e-7afd-4db0-b15a-72ebe9164bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:06.296542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf78e-7afd-4db0-b15a-72ebe9164bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:06.304334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9cf78e-7afd-4db0-b15a-72ebe9164bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:06.306152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9cf78e-7afd-4db0-b15a-72ebe9164bb4 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:06.315708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 763/1518] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:57:15.751450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77eb5-e772-4a9b-99e7-4c3a7f8999fb None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:15.751910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77eb5-e772-4a9b-99e7-4c3a7f8999fb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:15.752018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77eb5-e772-4a9b-99e7-4c3a7f8999fb None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:15.753333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 756/1519] 10.222.0.52 () {54 vars in 961 bytes} [Mon Oct 10 11:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:15.758132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f043ec-67cc-4d41-9a6f-df3df0252336 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:15.758577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f043ec-67cc-4d41-9a6f-df3df0252336 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:15.758723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f043ec-67cc-4d41-9a6f-df3df0252336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:15.779518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d9f043ec-67cc-4d41-9a6f-df3df0252336 None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:15.780700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d9f043ec-67cc-4d41-9a6f-df3df0252336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:15.789336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 764/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Oct 10 11:57:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:57:15.796886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae77eb5-e772-4a9b-99e7-4c3a7f8999fb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:15.810030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae77eb5-e772-4a9b-99e7-4c3a7f8999fb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:15.814948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a560a62-9c82-461c-bd70-f2550e2456b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:15.815527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560a62-9c82-461c-bd70-f2550e2456b4 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:15.815801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560a62-9c82-461c-bd70-f2550e2456b4 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:15.816067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560a62-9c82-461c-bd70-f2550e2456b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:15.824581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a560a62-9c82-461c-bd70-f2550e2456b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:15.826138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a560a62-9c82-461c-bd70-f2550e2456b4 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:15.835451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 757/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:57:17.773757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25549-d528-4718-af13-44ff6cc461bf None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:17.774273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25549-d528-4718-af13-44ff6cc461bf None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:17.774273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25549-d528-4718-af13-44ff6cc461bf None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:17.775589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:17.782087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ed73-249a-4b4f-88b5-33d47d8d6a1e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:17.782482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ed73-249a-4b4f-88b5-33d47d8d6a1e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:17.782653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ed73-249a-4b4f-88b5-33d47d8d6a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:17.802373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1931ed73-249a-4b4f-88b5-33d47d8d6a1e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:17.803598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1931ed73-249a-4b4f-88b5-33d47d8d6a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:17.811645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:57:17.965372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb25549-d528-4718-af13-44ff6cc461bf None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:17.980556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb25549-d528-4718-af13-44ff6cc461bf None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:17.985923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3621dd2-d7ce-4db4-905b-b4517a9f0c47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:17.986509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621dd2-d7ce-4db4-905b-b4517a9f0c47 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:17.986785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621dd2-d7ce-4db4-905b-b4517a9f0c47 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:17.987083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621dd2-d7ce-4db4-905b-b4517a9f0c47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:17.995644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3621dd2-d7ce-4db4-905b-b4517a9f0c47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:17.997206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3621dd2-d7ce-4db4-905b-b4517a9f0c47 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:18.006386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 766/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:57:19.061379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f733-706b-4ddc-b6eb-1f12ee3290c2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:19.061962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f733-706b-4ddc-b6eb-1f12ee3290c2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:19.061962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f733-706b-4ddc-b6eb-1f12ee3290c2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:19.063291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 11:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:19.069441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf637-f4b4-4b9c-af15-581800ce89f4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:19.069868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf637-f4b4-4b9c-af15-581800ce89f4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:19.069978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf637-f4b4-4b9c-af15-581800ce89f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:19.089426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c6ccf637-f4b4-4b9c-af15-581800ce89f4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:19.090609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c6ccf637-f4b4-4b9c-af15-581800ce89f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:19.098361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:57:19.250777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684f733-706b-4ddc-b6eb-1f12ee3290c2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:19.264905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684f733-706b-4ddc-b6eb-1f12ee3290c2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:19.270188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5624a0-95dd-48ea-8d81-bcd6e9850445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:19.270773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5624a0-95dd-48ea-8d81-bcd6e9850445 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:19.271081 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5624a0-95dd-48ea-8d81-bcd6e9850445 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:19.271352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5624a0-95dd-48ea-8d81-bcd6e9850445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:19.279713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5624a0-95dd-48ea-8d81-bcd6e9850445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:19.281271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5624a0-95dd-48ea-8d81-bcd6e9850445 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:19.290350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 760/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:57:23.706385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d794c0-027e-442b-8d92-c11c83b4e7fd None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:23.706979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d794c0-027e-442b-8d92-c11c83b4e7fd None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:23.706979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d794c0-027e-442b-8d92-c11c83b4e7fd None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:23.708224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:23.713577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4017dc-28aa-4dec-9d9b-a5ea25e2a315 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:23.713999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4017dc-28aa-4dec-9d9b-a5ea25e2a315 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:23.714127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4017dc-28aa-4dec-9d9b-a5ea25e2a315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:23.733232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dc4017dc-28aa-4dec-9d9b-a5ea25e2a315 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:23.734419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dc4017dc-28aa-4dec-9d9b-a5ea25e2a315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:23.742123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:57:24.034281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d794c0-027e-442b-8d92-c11c83b4e7fd None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:24.035994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c5d794c0-027e-442b-8d92-c11c83b4e7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:24.069470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:57:24.114619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d794c0-027e-442b-8d92-c11c83b4e7fd None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:24.118206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e1505-e34f-4baa-900f-a71b5d58af1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:24.118806 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e1505-e34f-4baa-900f-a71b5d58af1b service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:24.119266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e1505-e34f-4baa-900f-a71b5d58af1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:24.119535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e1505-e34f-4baa-900f-a71b5d58af1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:24.128901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e1505-e34f-4baa-900f-a71b5d58af1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:24.130352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e1505-e34f-4baa-900f-a71b5d58af1b service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:24.139535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 769/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:57:26.264530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f21bd-e1f0-4a03-adb2-ba04d2088105 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:26.265071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f21bd-e1f0-4a03-adb2-ba04d2088105 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:26.265071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f21bd-e1f0-4a03-adb2-ba04d2088105 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:26.266451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:26.271762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304695b1-8313-42a1-92b2-55219c379ba1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:26.272163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304695b1-8313-42a1-92b2-55219c379ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:26.272303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304695b1-8313-42a1-92b2-55219c379ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:26.291965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-304695b1-8313-42a1-92b2-55219c379ba1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:26.293148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-304695b1-8313-42a1-92b2-55219c379ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:26.301533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:57:26.591394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f21bd-e1f0-4a03-adb2-ba04d2088105 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:26.602034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f21bd-e1f0-4a03-adb2-ba04d2088105 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:26.606202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01683b-be3e-454d-93f0-aa21a3d4bea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:26.606791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01683b-be3e-454d-93f0-aa21a3d4bea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:26.607100 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01683b-be3e-454d-93f0-aa21a3d4bea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:26.607371 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01683b-be3e-454d-93f0-aa21a3d4bea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:26.616072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01683b-be3e-454d-93f0-aa21a3d4bea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:26.617490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01683b-be3e-454d-93f0-aa21a3d4bea5 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:26.626710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 763/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:57:30.294557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc46130-42ce-4720-bd7e-0b3f2179eef0 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:30.295129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc46130-42ce-4720-bd7e-0b3f2179eef0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:30.295248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc46130-42ce-4720-bd7e-0b3f2179eef0 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:30.296549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:30.301842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb10c7-d86d-4b12-b2ac-59653b6f9dfa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:30.302237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb10c7-d86d-4b12-b2ac-59653b6f9dfa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:30.302402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb10c7-d86d-4b12-b2ac-59653b6f9dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:30.323368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e4eb10c7-d86d-4b12-b2ac-59653b6f9dfa None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:30.324577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e4eb10c7-d86d-4b12-b2ac-59653b6f9dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:30.332498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:57:30.639360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc46130-42ce-4720-bd7e-0b3f2179eef0 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:30.641025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ecc46130-42ce-4720-bd7e-0b3f2179eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:30.717226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc46130-42ce-4720-bd7e-0b3f2179eef0 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:30.720753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f1e58-e2c4-487c-bca1-0d35ba8f7fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:30.721355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f1e58-e2c4-487c-bca1-0d35ba8f7fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:30.721652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f1e58-e2c4-487c-bca1-0d35ba8f7fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:30.721929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f1e58-e2c4-487c-bca1-0d35ba8f7fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:30.731204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f1e58-e2c4-487c-bca1-0d35ba8f7fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:30.732610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f1e58-e2c4-487c-bca1-0d35ba8f7fb1 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:30.742038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 772/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:57:32.886656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984071a-9837-4f62-bf5a-7a31981977c2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:32.887403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984071a-9837-4f62-bf5a-7a31981977c2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:32.887403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984071a-9837-4f62-bf5a-7a31981977c2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:32.888607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 11:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:57:32.894003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1afd9d-0ba5-4aff-929c-cc3bb6c5a505 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:32.894432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1afd9d-0ba5-4aff-929c-cc3bb6c5a505 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:32.894534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1afd9d-0ba5-4aff-929c-cc3bb6c5a505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:32.913608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bc1afd9d-0ba5-4aff-929c-cc3bb6c5a505 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:57:32.914784 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bc1afd9d-0ba5-4aff-929c-cc3bb6c5a505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:32.923048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:57:33.226524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984071a-9837-4f62-bf5a-7a31981977c2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:57:33.228214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2984071a-9837-4f62-bf5a-7a31981977c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:57:33.241431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984071a-9837-4f62-bf5a-7a31981977c2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:57:33.245489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b0f4f-198a-4f59-9a86-1e1db34e1a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:57:33.246073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0f4f-198a-4f59-9a86-1e1db34e1a7a service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:57:33.246510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0f4f-198a-4f59-9a86-1e1db34e1a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:57:33.246785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0f4f-198a-4f59-9a86-1e1db34e1a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:57:33.255527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b0f4f-198a-4f59-9a86-1e1db34e1a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:57:33.256931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b0f4f-198a-4f59-9a86-1e1db34e1a7a service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:57:33.266111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 766/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:57:42.908895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=63 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:57:43.262097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=37 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:57.153377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927dd03-c578-4902-9764-5488cb6728d6 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.154030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927dd03-c578-4902-9764-5488cb6728d6 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.154030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927dd03-c578-4902-9764-5488cb6728d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.180535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f927dd03-c578-4902-9764-5488cb6728d6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.181987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f927dd03-c578-4902-9764-5488cb6728d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.201768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 774/1540] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:57.211495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe59ed2-f48e-4b65-a88c-66be1cfa1816 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.213072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe59ed2-f48e-4b65-a88c-66be1cfa1816 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.213574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe59ed2-f48e-4b65-a88c-66be1cfa1816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.239468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8fe59ed2-f48e-4b65-a88c-66be1cfa1816 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.241048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8fe59ed2-f48e-4b65-a88c-66be1cfa1816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.251257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 767/1541] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:58:57.256231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927dd03-c578-4902-9764-5488cb6728d6 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:57.267133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927dd03-c578-4902-9764-5488cb6728d6 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:57.278332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:57.278973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.279258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.279515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.280652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:58:57.280958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:57.282677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:57.301512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 775/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:58:57.311387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe59ed2-f48e-4b65-a88c-66be1cfa1816 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:57.329985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe59ed2-f48e-4b65-a88c-66be1cfa1816 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:57.340170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbec2f5-a113-4638-9247-06087f2817ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:57.340817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbec2f5-a113-4638-9247-06087f2817ce admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.341337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbec2f5-a113-4638-9247-06087f2817ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.341550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbec2f5-a113-4638-9247-06087f2817ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.342346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbec2f5-a113-4638-9247-06087f2817ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:57.344540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbec2f5-a113-4638-9247-06087f2817ce admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:57.427666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 768/1543] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:57.432816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:57.447291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43231a-caf5-4521-bff6-e0d9df90cabd admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:57.522170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b5b98-9676-429e-a8ee-148d6e50c853 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.522528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b5b98-9676-429e-a8ee-148d6e50c853 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.522753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b5b98-9676-429e-a8ee-148d6e50c853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.542296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b075c92-ece5-46af-ba80-6e0598442462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:57.542908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b075c92-ece5-46af-ba80-6e0598442462 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.543194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b075c92-ece5-46af-ba80-6e0598442462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.543464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b075c92-ece5-46af-ba80-6e0598442462 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.544101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b075c92-ece5-46af-ba80-6e0598442462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:57.546670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b075c92-ece5-46af-ba80-6e0598442462 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:57.572703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-795b5b98-9676-429e-a8ee-148d6e50c853 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.573818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-795b5b98-9676-429e-a8ee-148d6e50c853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.591053 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 776/1544] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:57.593330 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 769/1545] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:58:57.595964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b075c92-ece5-46af-ba80-6e0598442462 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:57.602528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d7916-00c9-40c8-8e22-375261099fe4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.602807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d7916-00c9-40c8-8e22-375261099fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.603144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d7916-00c9-40c8-8e22-375261099fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.615717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b075c92-ece5-46af-ba80-6e0598442462 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:57.625775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684be29-ab6a-407f-a3eb-2287cbfa092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:57.626719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684be29-ab6a-407f-a3eb-2287cbfa092b admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.627258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684be29-ab6a-407f-a3eb-2287cbfa092b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.627767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684be29-ab6a-407f-a3eb-2287cbfa092b admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.628624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684be29-ab6a-407f-a3eb-2287cbfa092b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:57.630670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684be29-ab6a-407f-a3eb-2287cbfa092b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:57.642032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6f7d7916-00c9-40c8-8e22-375261099fe4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.643298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6f7d7916-00c9-40c8-8e22-375261099fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.650775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 777/1546] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:58:57.655330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684be29-ab6a-407f-a3eb-2287cbfa092b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:57.670737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684be29-ab6a-407f-a3eb-2287cbfa092b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:57.679046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e53e7-8a64-4a61-b8eb-ec0dc1b8230b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:57.680372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e53e7-8a64-4a61-b8eb-ec0dc1b8230b admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.681118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 770/1547] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:58:57.682222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e53e7-8a64-4a61-b8eb-ec0dc1b8230b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.682781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e53e7-8a64-4a61-b8eb-ec0dc1b8230b admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.683151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f61c-dbab-41d4-9997-ffaa290771d7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.683440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f61c-dbab-41d4-9997-ffaa290771d7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.683714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f61c-dbab-41d4-9997-ffaa290771d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.684430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e53e7-8a64-4a61-b8eb-ec0dc1b8230b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:57.686480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e53e7-8a64-4a61-b8eb-ec0dc1b8230b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:57.702940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 778/1548] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:58:57.704326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e53e7-8a64-4a61-b8eb-ec0dc1b8230b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:57.716238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e53e7-8a64-4a61-b8eb-ec0dc1b8230b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:57.725646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:57.726613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.727202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.727704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.727907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-63c5f61c-dbab-41d4-9997-ffaa290771d7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.729298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:58:57.729657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-63c5f61c-dbab-41d4-9997-ffaa290771d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.730041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:57.732105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:57.754654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 779/1549] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:58:57.755994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:57.766090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 771/1550] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:57.770946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f61c-dbab-41d4-9997-ffaa290771d7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:57.771662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bb786-c52d-4e11-8fcf-3de8d7a7508f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:57.777193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b37da9-1af3-4afe-92df-cc025168d745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:57.778131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37da9-1af3-4afe-92df-cc025168d745 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.778649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37da9-1af3-4afe-92df-cc025168d745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.779179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37da9-1af3-4afe-92df-cc025168d745 admin admin] PATH_INFO: `/v3/projects/0156957151444f94be28a65b4b018156/users/b051e414038b470691ee3e91b952a87d/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.779454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f61c-dbab-41d4-9997-ffaa290771d7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:57.793937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb40937-b141-405b-889e-8502432a9e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:57.795142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb40937-b141-405b-889e-8502432a9e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.795722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb40937-b141-405b-889e-8502432a9e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.796195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb40937-b141-405b-889e-8502432a9e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.799070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb40937-b141-405b-889e-8502432a9e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:57.801243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb40937-b141-405b-889e-8502432a9e76 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:57.811250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b37da9-1af3-4afe-92df-cc025168d745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0156957151444f94be28a65b4b018156, user_id=b051e414038b470691ee3e91b952a87d, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:57.812687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING py.warnings [None req-45b37da9-1af3-4afe-92df-cc025168d745 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:58:57.812687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: warnings.warn(msg) Oct 10 11:58:57.812687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: Oct 10 11:58:57.815263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b37da9-1af3-4afe-92df-cc025168d745 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:57.834674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 780/1551] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:58:57 2022] PUT /identity/v3/projects/0156957151444f94be28a65b4b018156/users/b051e414038b470691ee3e91b952a87d/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:58:57.836526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d898bc-90cc-49d5-ad1b-f994f2a69e78 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.837075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d898bc-90cc-49d5-ad1b-f994f2a69e78 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.837586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d898bc-90cc-49d5-ad1b-f994f2a69e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.840970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 772/1552] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:57.842378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5540cd-5ad6-440a-9a40-a79556e28b41 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.842721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5540cd-5ad6-440a-9a40-a79556e28b41 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.843103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5540cd-5ad6-440a-9a40-a79556e28b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.859537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a7d898bc-90cc-49d5-ad1b-f994f2a69e78 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.860785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a7d898bc-90cc-49d5-ad1b-f994f2a69e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.863273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5a5540cd-5ad6-440a-9a40-a79556e28b41 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.864533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5a5540cd-5ad6-440a-9a40-a79556e28b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.921822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 781/1553] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:57.923334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d69e-ceb8-4b18-95e7-77c1d8130bd2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.923615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d69e-ceb8-4b18-95e7-77c1d8130bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.923882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0d69e-ceb8-4b18-95e7-77c1d8130bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.938602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 773/1554] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:57.939898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183c232-1f51-4027-b23e-ada144136734 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.940232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183c232-1f51-4027-b23e-ada144136734 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.940519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183c232-1f51-4027-b23e-ada144136734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.944357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-38c0d69e-ceb8-4b18-95e7-77c1d8130bd2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.945593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-38c0d69e-ceb8-4b18-95e7-77c1d8130bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.953565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 782/1555] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:57.954731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8fc27-803c-4b3c-8df3-cffa230990fc None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.955048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8fc27-803c-4b3c-8df3-cffa230990fc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.955324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8fc27-803c-4b3c-8df3-cffa230990fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.963489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:57.967173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0183c232-1f51-4027-b23e-ada144136734 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.968405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0183c232-1f51-4027-b23e-ada144136734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.975179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d6d8fc27-803c-4b3c-8df3-cffa230990fc None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:57.976332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d6d8fc27-803c-4b3c-8df3-cffa230990fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:57.976994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 774/1556] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:57.978857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82de9f-9cce-4094-93a8-977470917ee3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:57.979166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82de9f-9cce-4094-93a8-977470917ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:57.979433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82de9f-9cce-4094-93a8-977470917ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:57.995509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 783/1557] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:57.995509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8fc27-803c-4b3c-8df3-cffa230990fc None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.001555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cc82de9f-9cce-4094-93a8-977470917ee3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:58.002584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8fc27-803c-4b3c-8df3-cffa230990fc None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.002738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cc82de9f-9cce-4094-93a8-977470917ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:58.007371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa247390-c95d-4146-9fe7-fd559412297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.007988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa247390-c95d-4146-9fe7-fd559412297d admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.008306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa247390-c95d-4146-9fe7-fd559412297d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.008532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa247390-c95d-4146-9fe7-fd559412297d admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.009199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa247390-c95d-4146-9fe7-fd559412297d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.010348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 775/1558] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:58.013937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa247390-c95d-4146-9fe7-fd559412297d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.014091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a7a34-363e-4582-8378-b4e8a2ab2203 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.014091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a7a34-363e-4582-8378-b4e8a2ab2203 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.014091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a7a34-363e-4582-8378-b4e8a2ab2203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.033677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8d6a7a34-363e-4582-8378-b4e8a2ab2203 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:58.034993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8d6a7a34-363e-4582-8378-b4e8a2ab2203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:58.039600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 784/1559] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:58:57 2022] POST /identity/v3/users => generated 286 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.039600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522baee-b4c8-48e0-9897-ca51cfeaf9c7 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.039600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522baee-b4c8-48e0-9897-ca51cfeaf9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.039600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522baee-b4c8-48e0-9897-ca51cfeaf9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.044404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 776/1560] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:58.045186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a7a34-363e-4582-8378-b4e8a2ab2203 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.056601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a7a34-363e-4582-8378-b4e8a2ab2203 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.061138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c522baee-b4c8-48e0-9897-ca51cfeaf9c7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:58.061259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706105a4-f71c-4e37-98a6-a1cc61a62448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.061929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706105a4-f71c-4e37-98a6-a1cc61a62448 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.062173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706105a4-f71c-4e37-98a6-a1cc61a62448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.062359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c522baee-b4c8-48e0-9897-ca51cfeaf9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:58.062458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706105a4-f71c-4e37-98a6-a1cc61a62448 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.063116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706105a4-f71c-4e37-98a6-a1cc61a62448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:58:58.063423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706105a4-f71c-4e37-98a6-a1cc61a62448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.064426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-706105a4-f71c-4e37-98a6-a1cc61a62448 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:58:58.064426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Oct 10 11:58:58.064426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: Oct 10 11:58:58.065305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706105a4-f71c-4e37-98a6-a1cc61a62448 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.070980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 785/1561] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:58.072205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f3855-bc28-4ca0-aea5-e8cfc90062d5 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.072478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f3855-bc28-4ca0-aea5-e8cfc90062d5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.072752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f3855-bc28-4ca0-aea5-e8cfc90062d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.086935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 777/1562] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:58:58.086935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322983f-37f6-446d-961a-7da2abb10552 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.086935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322983f-37f6-446d-961a-7da2abb10552 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.086935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322983f-37f6-446d-961a-7da2abb10552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.099608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9f8f3855-bc28-4ca0-aea5-e8cfc90062d5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:58.100819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9f8f3855-bc28-4ca0-aea5-e8cfc90062d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:58.101945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2322983f-37f6-446d-961a-7da2abb10552 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:58.103770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2322983f-37f6-446d-961a-7da2abb10552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:58.109607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 786/1563] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:58.110760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83f975-3591-4929-8f8b-5775023c4986 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.111084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83f975-3591-4929-8f8b-5775023c4986 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.111361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83f975-3591-4929-8f8b-5775023c4986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.114607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 778/1564] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:58.115534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322983f-37f6-446d-961a-7da2abb10552 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.131561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3b83f975-3591-4929-8f8b-5775023c4986 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:58.132818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3b83f975-3591-4929-8f8b-5775023c4986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:58.140853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 787/1565] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:58.141613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83f975-3591-4929-8f8b-5775023c4986 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.152258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83f975-3591-4929-8f8b-5775023c4986 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.156988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d118257-c5da-427d-88f3-6a9ed0083068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.157602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d118257-c5da-427d-88f3-6a9ed0083068 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.157848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d118257-c5da-427d-88f3-6a9ed0083068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.158114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d118257-c5da-427d-88f3-6a9ed0083068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.158747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d118257-c5da-427d-88f3-6a9ed0083068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.160369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d118257-c5da-427d-88f3-6a9ed0083068 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.164549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:58.171357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 788/1566] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:58.175395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e73b4e-1c5b-46be-ac2f-aafdfafefe7e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.175676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e73b4e-1c5b-46be-ac2f-aafdfafefe7e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.175950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e73b4e-1c5b-46be-ac2f-aafdfafefe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.178017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322983f-37f6-446d-961a-7da2abb10552 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.181969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb595d9-86cd-43e5-a4a6-0b2f6ed26921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.182574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb595d9-86cd-43e5-a4a6-0b2f6ed26921 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.182881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb595d9-86cd-43e5-a4a6-0b2f6ed26921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.183180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb595d9-86cd-43e5-a4a6-0b2f6ed26921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.192286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb595d9-86cd-43e5-a4a6-0b2f6ed26921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.193712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb595d9-86cd-43e5-a4a6-0b2f6ed26921 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.195670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-01e73b4e-1c5b-46be-ac2f-aafdfafefe7e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:58.196859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-01e73b4e-1c5b-46be-ac2f-aafdfafefe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:58.203167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 779/1567] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:58:58.203950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb595d9-86cd-43e5-a4a6-0b2f6ed26921 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.208280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 789/1568] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:58.208280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e73b4e-1c5b-46be-ac2f-aafdfafefe7e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.213648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb595d9-86cd-43e5-a4a6-0b2f6ed26921 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.219422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e73b4e-1c5b-46be-ac2f-aafdfafefe7e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.219869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.220487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.220885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.221104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.221749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:58:58.222064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.223674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.225383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.225980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.226274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.226592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.227251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.228816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING py.warnings [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:58:58.228816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: warnings.warn(msg) Oct 10 11:58:58.228816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: Oct 10 11:58:58.230057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.238535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 780/1569] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:58:58.238535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.249194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe57f-d0d0-4e76-b0c4-2bf568e00f9a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.253991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.254607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.254909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.255187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.255827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:58:58.256124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.257851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.270203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 790/1570] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.271281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 781/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:58:58.272041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.272314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.286493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c19a9-95c1-4e18-97e1-ce564ab2b5ad admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.348419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.349060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.349312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.349584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.350284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:58:58.350587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.352236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.359689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050cdcf-a1dd-4f3f-ad3b-5a577b159232 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.365113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 782/1572] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:58:58.367372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.367997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.368279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.368813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.369468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:58:58.369770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.371543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.372394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.388059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 791/1573] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:58:58.388167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b717c-b44f-4b2e-8d84-1cf9a7e4648b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.388248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.392875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f099a1-5f17-41e3-86b6-d88f487fbbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.394121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f099a1-5f17-41e3-86b6-d88f487fbbdc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.398618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dce03c-6c24-49dc-9717-3edd87e7fb50 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.399312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f099a1-5f17-41e3-86b6-d88f487fbbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.400108 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f099a1-5f17-41e3-86b6-d88f487fbbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.401548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f099a1-5f17-41e3-86b6-d88f487fbbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.403334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f099a1-5f17-41e3-86b6-d88f487fbbdc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.408926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.410101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.410492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.410834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.412373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:58:58.412373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.415685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.420380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 783/1574] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:58.421515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f099a1-5f17-41e3-86b6-d88f487fbbdc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.439001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 792/1575] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:58:58.439001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.439304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f099a1-5f17-41e3-86b6-d88f487fbbdc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.443534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f72143-858a-4879-85a8-1d5212053889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.444174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f72143-858a-4879-85a8-1d5212053889 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.444456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f72143-858a-4879-85a8-1d5212053889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.444729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f72143-858a-4879-85a8-1d5212053889 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.445501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f72143-858a-4879-85a8-1d5212053889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.447925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f72143-858a-4879-85a8-1d5212053889 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.451852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0be37e-fc76-4531-af22-ca05d9ba7f8b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.458711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ec55d-b54b-4c1f-8039-5a457dfd6d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.460098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ec55d-b54b-4c1f-8039-5a457dfd6d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.461068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ec55d-b54b-4c1f-8039-5a457dfd6d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.461316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ec55d-b54b-4c1f-8039-5a457dfd6d2d admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.462039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ec55d-b54b-4c1f-8039-5a457dfd6d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.464905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ec55d-b54b-4c1f-8039-5a457dfd6d2d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.476484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 784/1576] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.477185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f72143-858a-4879-85a8-1d5212053889 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.497056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 793/1577] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/users => generated 290 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.497056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ec55d-b54b-4c1f-8039-5a457dfd6d2d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.510456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ec55d-b54b-4c1f-8039-5a457dfd6d2d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.548840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f72143-858a-4879-85a8-1d5212053889 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.556741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ac79e-18c8-4642-be36-69a9e37873e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.558148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac79e-18c8-4642-be36-69a9e37873e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.558651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac79e-18c8-4642-be36-69a9e37873e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.559066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac79e-18c8-4642-be36-69a9e37873e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.559826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ac79e-18c8-4642-be36-69a9e37873e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.562288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ac79e-18c8-4642-be36-69a9e37873e4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.571015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:58.591967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dded37-c4cd-4782-a04d-80521eaa588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.592820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dded37-c4cd-4782-a04d-80521eaa588a admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.593121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dded37-c4cd-4782-a04d-80521eaa588a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.593498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dded37-c4cd-4782-a04d-80521eaa588a admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.594381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dded37-c4cd-4782-a04d-80521eaa588a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.596460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dded37-c4cd-4782-a04d-80521eaa588a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.597877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 785/1578] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/projects => generated 393 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.598519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ac79e-18c8-4642-be36-69a9e37873e4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.609806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:58.641559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 794/1579] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.642388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dded37-c4cd-4782-a04d-80521eaa588a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.704159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ac79e-18c8-4642-be36-69a9e37873e4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.732793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dded37-c4cd-4782-a04d-80521eaa588a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.737913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49509b9a-2711-4c09-a019-b272cf43631e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.739007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509b9a-2711-4c09-a019-b272cf43631e admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.739290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509b9a-2711-4c09-a019-b272cf43631e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.739723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509b9a-2711-4c09-a019-b272cf43631e admin admin] PATH_INFO: `/v3/projects/41c3718a446542109d01276a99950b84/users/f633c812bb6446a49107a05b3c956366/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.756299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49509b9a-2711-4c09-a019-b272cf43631e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c3718a446542109d01276a99950b84, user_id=f633c812bb6446a49107a05b3c956366, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.759063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49509b9a-2711-4c09-a019-b272cf43631e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.772030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b185c8-0485-4b84-b2dd-07474c66b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.772701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b185c8-0485-4b84-b2dd-07474c66b76b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.772985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b185c8-0485-4b84-b2dd-07474c66b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.773196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 795/1580] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:58:58 2022] PUT /identity/v3/projects/41c3718a446542109d01276a99950b84/users/f633c812bb6446a49107a05b3c956366/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:58:58.773336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b185c8-0485-4b84-b2dd-07474c66b76b admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.774048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49509b9a-2711-4c09-a019-b272cf43631e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.774157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b185c8-0485-4b84-b2dd-07474c66b76b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.776737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b185c8-0485-4b84-b2dd-07474c66b76b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.804923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 786/1581] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.806094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b185c8-0485-4b84-b2dd-07474c66b76b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.856735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49509b9a-2711-4c09-a019-b272cf43631e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.885368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b185c8-0485-4b84-b2dd-07474c66b76b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.891195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.892041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.892327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.892603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.893244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.900139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:58:58.900139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Oct 10 11:58:58.900139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: Oct 10 11:58:58.901819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.930587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 787/1582] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/users => generated 278 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.931991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.935964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67e18d-f8dd-4ef3-910b-a44e42e35deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.937155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e18d-f8dd-4ef3-910b-a44e42e35deb admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.938332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e18d-f8dd-4ef3-910b-a44e42e35deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.939061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e18d-f8dd-4ef3-910b-a44e42e35deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.939926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67e18d-f8dd-4ef3-910b-a44e42e35deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.942289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67e18d-f8dd-4ef3-910b-a44e42e35deb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.944075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dac497-de94-4be3-ad34-c549a739c981 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:58.948503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:58.952316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717a84d-b7d6-4342-ac98-4c0d953f7229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:58.955123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717a84d-b7d6-4342-ac98-4c0d953f7229 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:58.955695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717a84d-b7d6-4342-ac98-4c0d953f7229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:58.955974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717a84d-b7d6-4342-ac98-4c0d953f7229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:58.958342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717a84d-b7d6-4342-ac98-4c0d953f7229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:58.960198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717a84d-b7d6-4342-ac98-4c0d953f7229 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:58.982650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 788/1583] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:58.982650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717a84d-b7d6-4342-ac98-4c0d953f7229 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.996862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 796/1584] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/projects => generated 401 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:58.998089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67e18d-f8dd-4ef3-910b-a44e42e35deb admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:58.998287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717a84d-b7d6-4342-ac98-4c0d953f7229 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.012446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:59.075839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67e18d-f8dd-4ef3-910b-a44e42e35deb admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.080581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff7e7-a1aa-4969-98ae-6a5541e77f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.081757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff7e7-a1aa-4969-98ae-6a5541e77f9c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.082043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff7e7-a1aa-4969-98ae-6a5541e77f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.082318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff7e7-a1aa-4969-98ae-6a5541e77f9c admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.083643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ff7e7-a1aa-4969-98ae-6a5541e77f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.084883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f44e68-0c83-4145-a0c1-2f22bde4748a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.085683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f44e68-0c83-4145-a0c1-2f22bde4748a admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.085967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f44e68-0c83-4145-a0c1-2f22bde4748a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.086239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f44e68-0c83-4145-a0c1-2f22bde4748a admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.086865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f44e68-0c83-4145-a0c1-2f22bde4748a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.087330 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ff7e7-a1aa-4969-98ae-6a5541e77f9c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.089990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f44e68-0c83-4145-a0c1-2f22bde4748a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.117098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 797/1585] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:59.118370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeaa42-2e52-4f45-8b98-2e73dac6a827 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.119379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeaa42-2e52-4f45-8b98-2e73dac6a827 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.119599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeaa42-2e52-4f45-8b98-2e73dac6a827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.119916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 789/1586] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:58:58 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:59.121212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff7e7-a1aa-4969-98ae-6a5541e77f9c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.133482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff7e7-a1aa-4969-98ae-6a5541e77f9c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.143318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e659-26fe-44ae-b771-658050f3ae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.144105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e659-26fe-44ae-b771-658050f3ae60 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.144429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e659-26fe-44ae-b771-658050f3ae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.144737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e659-26fe-44ae-b771-658050f3ae60 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.145528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640e659-26fe-44ae-b771-658050f3ae60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.147904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640e659-26fe-44ae-b771-658050f3ae60 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.164634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6bfeaa42-2e52-4f45-8b98-2e73dac6a827 None None] MFA Rules not processed for user `f633c812bb6446a49107a05b3c956366`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:59.166525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6bfeaa42-2e52-4f45-8b98-2e73dac6a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:59.183660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 790/1587] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:58:59 2022] POST /identity/v3/users => generated 293 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:59.184735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e659-26fe-44ae-b771-658050f3ae60 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.196453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e659-26fe-44ae-b771-658050f3ae60 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.202167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5807a2a-638b-47d6-b946-2d085d2155de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.203126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5807a2a-638b-47d6-b946-2d085d2155de admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.203463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5807a2a-638b-47d6-b946-2d085d2155de admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.203916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5807a2a-638b-47d6-b946-2d085d2155de admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.204559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5807a2a-638b-47d6-b946-2d085d2155de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.207417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5807a2a-638b-47d6-b946-2d085d2155de admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.225126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 791/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.229104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5807a2a-638b-47d6-b946-2d085d2155de admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.240658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5807a2a-638b-47d6-b946-2d085d2155de admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.247645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbec3d-68b2-4b70-ad52-e412a4fb086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.248297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 798/1589] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:59.249077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeaa42-2e52-4f45-8b98-2e73dac6a827 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.251299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbec3d-68b2-4b70-ad52-e412a4fb086e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.251835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbec3d-68b2-4b70-ad52-e412a4fb086e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.252317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbec3d-68b2-4b70-ad52-e412a4fb086e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.253274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbec3d-68b2-4b70-ad52-e412a4fb086e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.256458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbec3d-68b2-4b70-ad52-e412a4fb086e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.264079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeaa42-2e52-4f45-8b98-2e73dac6a827 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.270058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141884-0464-4d33-a42b-1c7b3ee210b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.270652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141884-0464-4d33-a42b-1c7b3ee210b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.270950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141884-0464-4d33-a42b-1c7b3ee210b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.271222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141884-0464-4d33-a42b-1c7b3ee210b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.271846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32141884-0464-4d33-a42b-1c7b3ee210b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.273399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 792/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.274151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32141884-0464-4d33-a42b-1c7b3ee210b2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.278462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbec3d-68b2-4b70-ad52-e412a4fb086e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.291223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbec3d-68b2-4b70-ad52-e412a4fb086e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.297996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8999bd-ef08-4c6c-a77a-afbc0b987ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.299258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8999bd-ef08-4c6c-a77a-afbc0b987ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.299703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8999bd-ef08-4c6c-a77a-afbc0b987ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.300156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8999bd-ef08-4c6c-a77a-afbc0b987ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.300853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8999bd-ef08-4c6c-a77a-afbc0b987ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.303009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8999bd-ef08-4c6c-a77a-afbc0b987ee9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.310034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 799/1591] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:58:59 2022] POST /identity/v3/users => generated 301 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:58:59.315339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141884-0464-4d33-a42b-1c7b3ee210b2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.319602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 793/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.320306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8999bd-ef08-4c6c-a77a-afbc0b987ee9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.326789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141884-0464-4d33-a42b-1c7b3ee210b2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.332656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79efe20-892a-4cd0-bc7d-74ed15831410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.333488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efe20-892a-4cd0-bc7d-74ed15831410 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.333770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efe20-892a-4cd0-bc7d-74ed15831410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.333976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8999bd-ef08-4c6c-a77a-afbc0b987ee9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.334186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79efe20-892a-4cd0-bc7d-74ed15831410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.334909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79efe20-892a-4cd0-bc7d-74ed15831410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.336994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79efe20-892a-4cd0-bc7d-74ed15831410 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.339437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68635ae-d68e-4488-bddc-4055708afe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.340050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68635ae-d68e-4488-bddc-4055708afe8f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.340890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68635ae-d68e-4488-bddc-4055708afe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.341265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68635ae-d68e-4488-bddc-4055708afe8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.342016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68635ae-d68e-4488-bddc-4055708afe8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.343899 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68635ae-d68e-4488-bddc-4055708afe8f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.354732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 800/1593] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.356082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79efe20-892a-4cd0-bc7d-74ed15831410 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.370002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 794/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.370002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68635ae-d68e-4488-bddc-4055708afe8f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.370751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79efe20-892a-4cd0-bc7d-74ed15831410 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.377672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095041e6-c479-4b06-b288-afd9afd4ffe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.378620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095041e6-c479-4b06-b288-afd9afd4ffe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.379129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095041e6-c479-4b06-b288-afd9afd4ffe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.379515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095041e6-c479-4b06-b288-afd9afd4ffe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.379752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68635ae-d68e-4488-bddc-4055708afe8f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.380292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095041e6-c479-4b06-b288-afd9afd4ffe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.382000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095041e6-c479-4b06-b288-afd9afd4ffe8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.386599 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667176a-8f88-4167-b0cd-557039c658a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.387559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667176a-8f88-4167-b0cd-557039c658a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.388471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667176a-8f88-4167-b0cd-557039c658a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.388931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667176a-8f88-4167-b0cd-557039c658a2 admin admin] PATH_INFO: `/v3/projects/d57fdf43bdef4f0dabd51175a84e9093/users/8568f2501f824875bae7ab1825cc2c32/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.393832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 801/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.395380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604717a-3c46-4a9b-b022-982b29b3e040 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.395719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604717a-3c46-4a9b-b022-982b29b3e040 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.396044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604717a-3c46-4a9b-b022-982b29b3e040 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.397687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 802/1596] 10.222.0.52 () {54 vars in 961 bytes} [Mon Oct 10 11:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:58:59.398828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604717a-3c46-4a9b-b022-982b29b3e040 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.415333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604717a-3c46-4a9b-b022-982b29b3e040 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.419969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667176a-8f88-4167-b0cd-557039c658a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57fdf43bdef4f0dabd51175a84e9093, user_id=8568f2501f824875bae7ab1825cc2c32, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.422913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667176a-8f88-4167-b0cd-557039c658a2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.427218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b7db0-ad01-44ee-bfcb-c07dbc0c01de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.427218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7db0-ad01-44ee-bfcb-c07dbc0c01de admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.427218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7db0-ad01-44ee-bfcb-c07dbc0c01de admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.427218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7db0-ad01-44ee-bfcb-c07dbc0c01de admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.427783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b7db0-ad01-44ee-bfcb-c07dbc0c01de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.430109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b7db0-ad01-44ee-bfcb-c07dbc0c01de admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.442939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 803/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.444027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b7db0-ad01-44ee-bfcb-c07dbc0c01de admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.447105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 795/1598] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:58:59 2022] PUT /identity/v3/projects/d57fdf43bdef4f0dabd51175a84e9093/users/8568f2501f824875bae7ab1825cc2c32/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:58:59.450035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667176a-8f88-4167-b0cd-557039c658a2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.463358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b7db0-ad01-44ee-bfcb-c07dbc0c01de admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.477001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:59.550283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667176a-8f88-4167-b0cd-557039c658a2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.556806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5069-609f-41b3-9945-838a7c35d0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.557438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5069-609f-41b3-9945-838a7c35d0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.557856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5069-609f-41b3-9945-838a7c35d0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.558134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5069-609f-41b3-9945-838a7c35d0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.558995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d5069-609f-41b3-9945-838a7c35d0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.560625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d5069-609f-41b3-9945-838a7c35d0aa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.576959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 796/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.576959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5069-609f-41b3-9945-838a7c35d0aa admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.581543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ae09-d73d-4b44-804b-c9c8e364c109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.582561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ae09-d73d-4b44-804b-c9c8e364c109 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.582856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ae09-d73d-4b44-804b-c9c8e364c109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.583151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ae09-d73d-4b44-804b-c9c8e364c109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.584471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454ae09-d73d-4b44-804b-c9c8e364c109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.586868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454ae09-d73d-4b44-804b-c9c8e364c109 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.589591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5069-609f-41b3-9945-838a7c35d0aa admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.597131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c38b1-e350-4d81-a00b-1421921bb73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.597870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c38b1-e350-4d81-a00b-1421921bb73a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.598718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c38b1-e350-4d81-a00b-1421921bb73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.599245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c38b1-e350-4d81-a00b-1421921bb73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.600261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c38b1-e350-4d81-a00b-1421921bb73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.604198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 804/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.604366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c38b1-e350-4d81-a00b-1421921bb73a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.604465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ae09-d73d-4b44-804b-c9c8e364c109 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.616348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 797/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.617741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d156-e038-460b-bff9-d12fc554c891 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.617885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ae09-d73d-4b44-804b-c9c8e364c109 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.618026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d156-e038-460b-bff9-d12fc554c891 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.618393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d156-e038-460b-bff9-d12fc554c891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.624918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad4a3-f4cb-47db-b593-3dc57eb4fcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.625509 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad4a3-f4cb-47db-b593-3dc57eb4fcd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.625786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad4a3-f4cb-47db-b593-3dc57eb4fcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.626055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad4a3-f4cb-47db-b593-3dc57eb4fcd2 admin admin] PATH_INFO: `/v3/projects/4293c37dca19479297f1e60de20a865f/users/55d073e4d9af4c7f923889350ba74cda/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.631169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:59.657153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ad4a3-f4cb-47db-b593-3dc57eb4fcd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4293c37dca19479297f1e60de20a865f, user_id=55d073e4d9af4c7f923889350ba74cda, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.660093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ad4a3-f4cb-47db-b593-3dc57eb4fcd2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.668387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9b04d156-e038-460b-bff9-d12fc554c891 None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:59.670702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9b04d156-e038-460b-bff9-d12fc554c891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:59.678858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 805/1602] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:58:59 2022] PUT /identity/v3/projects/4293c37dca19479297f1e60de20a865f/users/55d073e4d9af4c7f923889350ba74cda/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:58:59.680104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad4a3-f4cb-47db-b593-3dc57eb4fcd2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.766348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad4a3-f4cb-47db-b593-3dc57eb4fcd2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.773160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5a423-3be2-4aeb-912a-8270ee149f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.773916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5a423-3be2-4aeb-912a-8270ee149f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.774250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5a423-3be2-4aeb-912a-8270ee149f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.774576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5a423-3be2-4aeb-912a-8270ee149f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.775406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5a423-3be2-4aeb-912a-8270ee149f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.777118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5a423-3be2-4aeb-912a-8270ee149f91 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.784135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 798/1603] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Oct 10 11:58:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:59.801630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 806/1604] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:58:59.801630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5a423-3be2-4aeb-912a-8270ee149f91 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.805429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f29abf-457a-457c-aa9e-d8e69eaab3ce None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.806013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f29abf-457a-457c-aa9e-d8e69eaab3ce None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.807445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f29abf-457a-457c-aa9e-d8e69eaab3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.811353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5a423-3be2-4aeb-912a-8270ee149f91 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.816466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83bcea-1989-45c2-9737-f1f4ec42243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.817178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83bcea-1989-45c2-9737-f1f4ec42243d admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.817459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83bcea-1989-45c2-9737-f1f4ec42243d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.817729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83bcea-1989-45c2-9737-f1f4ec42243d admin admin] PATH_INFO: `/v3/projects/b72dcfa7755e499f8f55ade0fde3d7c7/users/99b3e727e6fa480c90a054a60b68b8d0/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.832076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-87f29abf-457a-457c-aa9e-d8e69eaab3ce None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:58:59.833341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-87f29abf-457a-457c-aa9e-d8e69eaab3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:58:59.833831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83bcea-1989-45c2-9737-f1f4ec42243d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72dcfa7755e499f8f55ade0fde3d7c7, user_id=99b3e727e6fa480c90a054a60b68b8d0, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.836769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83bcea-1989-45c2-9737-f1f4ec42243d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.846016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 799/1605] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:58:59.847347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f29abf-457a-457c-aa9e-d8e69eaab3ce None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.855776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 807/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:58:59 2022] PUT /identity/v3/projects/b72dcfa7755e499f8f55ade0fde3d7c7/users/99b3e727e6fa480c90a054a60b68b8d0/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:58:59.856950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83bcea-1989-45c2-9737-f1f4ec42243d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:58:59.864034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f29abf-457a-457c-aa9e-d8e69eaab3ce None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.937753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:58:59.941175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d3ab6-f1f3-4a71-9488-2bb975b7cd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.941864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d3ab6-f1f3-4a71-9488-2bb975b7cd97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.942144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d3ab6-f1f3-4a71-9488-2bb975b7cd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.942506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d3ab6-f1f3-4a71-9488-2bb975b7cd97 admin admin] PATH_INFO: `/v3/projects/79aee39298624f99b33b9808fe5030f8/users/0c0f1d116c004694b33b19d988ac4959/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.943132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83bcea-1989-45c2-9737-f1f4ec42243d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:58:59.950325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529e8d9-d572-4545-a673-d7a2a5fdddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:58:59.951651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529e8d9-d572-4545-a673-d7a2a5fdddc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.952264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529e8d9-d572-4545-a673-d7a2a5fdddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.952663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529e8d9-d572-4545-a673-d7a2a5fdddc9 admin admin] PATH_INFO: `/v3/projects/de3e893bfb574016a6e5e020f96490c5/users/2f7735444ca8431ea7906070374dbe67/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.961427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d3ab6-f1f3-4a71-9488-2bb975b7cd97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79aee39298624f99b33b9808fe5030f8, user_id=0c0f1d116c004694b33b19d988ac4959, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.964644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d3ab6-f1f3-4a71-9488-2bb975b7cd97 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.973047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529e8d9-d572-4545-a673-d7a2a5fdddc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3e893bfb574016a6e5e020f96490c5, user_id=2f7735444ca8431ea7906070374dbe67, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:58:59.976325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529e8d9-d572-4545-a673-d7a2a5fdddc9 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:58:59.986986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 800/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:58:59 2022] PUT /identity/v3/projects/79aee39298624f99b33b9808fe5030f8/users/0c0f1d116c004694b33b19d988ac4959/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:58:59.996515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291fae-8f96-4c3d-baad-42cecd66e9ec None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:58:59.996816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291fae-8f96-4c3d-baad-42cecd66e9ec None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:58:59.997095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291fae-8f96-4c3d-baad-42cecd66e9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:58:59.997603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 808/1608] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:58:59 2022] PUT /identity/v3/projects/de3e893bfb574016a6e5e020f96490c5/users/2f7735444ca8431ea7906070374dbe67/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:58:59.998468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529e8d9-d572-4545-a673-d7a2a5fdddc9 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.061362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bd291fae-8f96-4c3d-baad-42cecd66e9ec None None] MFA Rules not processed for user `55d073e4d9af4c7f923889350ba74cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:00.061362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bd291fae-8f96-4c3d-baad-42cecd66e9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:00.136656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529e8d9-d572-4545-a673-d7a2a5fdddc9 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.143125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:00.145757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379714ac-6999-4eac-83d7-bf90e04ab5c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.146561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379714ac-6999-4eac-83d7-bf90e04ab5c6 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.146849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379714ac-6999-4eac-83d7-bf90e04ab5c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.147229 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379714ac-6999-4eac-83d7-bf90e04ab5c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.172199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 801/1609] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:58:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:00.174877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd291fae-8f96-4c3d-baad-42cecd66e9ec None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.220030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379714ac-6999-4eac-83d7-bf90e04ab5c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.221807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379714ac-6999-4eac-83d7-bf90e04ab5c6 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.233836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 809/1610] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:58:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:00.236583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fbeb1-5e63-438f-afac-1bb6a77caad1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.236904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fbeb1-5e63-438f-afac-1bb6a77caad1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.237249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fbeb1-5e63-438f-afac-1bb6a77caad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.260463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd291fae-8f96-4c3d-baad-42cecd66e9ec None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.266776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-867fbeb1-5e63-438f-afac-1bb6a77caad1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:00.266973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6077e0-5392-4610-ac7f-88835757de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.268192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-867fbeb1-5e63-438f-afac-1bb6a77caad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:00.268434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6077e0-5392-4610-ac7f-88835757de35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.268869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6077e0-5392-4610-ac7f-88835757de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.269297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6077e0-5392-4610-ac7f-88835757de35 admin admin] PATH_INFO: `/v3/projects/830b3f1b187e4811a3180a5f07490691/users/e591a4012c0b4b7fa46dc1ef72958a68/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.280902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 810/1611] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:00.281982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fbeb1-5e63-438f-afac-1bb6a77caad1 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.288942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-867fbeb1-5e63-438f-afac-1bb6a77caad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:00.294088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6077e0-5392-4610-ac7f-88835757de35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830b3f1b187e4811a3180a5f07490691, user_id=e591a4012c0b4b7fa46dc1ef72958a68, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.297820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6077e0-5392-4610-ac7f-88835757de35 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.322564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 802/1612] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:00 2022] PUT /identity/v3/projects/830b3f1b187e4811a3180a5f07490691/users/e591a4012c0b4b7fa46dc1ef72958a68/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:00.324146 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b2f1-0ae0-4e17-96c4-f65a070bd3ab None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.324430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b2f1-0ae0-4e17-96c4-f65a070bd3ab None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.324728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b2f1-0ae0-4e17-96c4-f65a070bd3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.350288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-26e1b2f1-0ae0-4e17-96c4-f65a070bd3ab None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:00.351553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-26e1b2f1-0ae0-4e17-96c4-f65a070bd3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:00.392857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fbeb1-5e63-438f-afac-1bb6a77caad1 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.440450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 803/1613] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:00.441215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:00.442120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942069c-f4ce-4a81-bb38-0a5b0480e568 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.442422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942069c-f4ce-4a81-bb38-0a5b0480e568 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.442801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942069c-f4ce-4a81-bb38-0a5b0480e568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.482720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e942069c-f4ce-4a81-bb38-0a5b0480e568 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:00.484445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e942069c-f4ce-4a81-bb38-0a5b0480e568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:00.484915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca976c-ff15-4ad7-a04c-61051361f890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.485817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca976c-ff15-4ad7-a04c-61051361f890 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.486199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca976c-ff15-4ad7-a04c-61051361f890 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.486557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca976c-ff15-4ad7-a04c-61051361f890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.493210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 804/1614] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:00.494229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942069c-f4ce-4a81-bb38-0a5b0480e568 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.500524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca976c-ff15-4ad7-a04c-61051361f890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.502945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca976c-ff15-4ad7-a04c-61051361f890 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.513566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 811/1615] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:00.514684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca976c-ff15-4ad7-a04c-61051361f890 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.521275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942069c-f4ce-4a81-bb38-0a5b0480e568 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.524754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca976c-ff15-4ad7-a04c-61051361f890 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.527355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6735c7c-4478-4409-91b3-81c889079a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.528249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6735c7c-4478-4409-91b3-81c889079a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.528732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6735c7c-4478-4409-91b3-81c889079a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.529203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6735c7c-4478-4409-91b3-81c889079a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.529312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb69920-245b-4a9f-9f53-5f3c642e83c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.530295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6735c7c-4478-4409-91b3-81c889079a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.530497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb69920-245b-4a9f-9f53-5f3c642e83c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.530788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb69920-245b-4a9f-9f53-5f3c642e83c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.531100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb69920-245b-4a9f-9f53-5f3c642e83c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.533236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6735c7c-4478-4409-91b3-81c889079a55 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.543804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb69920-245b-4a9f-9f53-5f3c642e83c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.545372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb69920-245b-4a9f-9f53-5f3c642e83c3 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.557280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 812/1616] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:00.558640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677fb7e-e2dc-4538-9548-9b5953dbcd66 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.558951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677fb7e-e2dc-4538-9548-9b5953dbcd66 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.559296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677fb7e-e2dc-4538-9548-9b5953dbcd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.584200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9677fb7e-e2dc-4538-9548-9b5953dbcd66 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:00.585434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9677fb7e-e2dc-4538-9548-9b5953dbcd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:00.586216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 805/1617] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:00 2022] POST /identity/v3/projects => generated 357 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:00.587111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6735c7c-4478-4409-91b3-81c889079a55 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.665317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:00.692679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6735c7c-4478-4409-91b3-81c889079a55 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.705374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527909a6-79aa-44bb-b4dd-0faa84d4a64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.706025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527909a6-79aa-44bb-b4dd-0faa84d4a64b service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.706304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527909a6-79aa-44bb-b4dd-0faa84d4a64b service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.706677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527909a6-79aa-44bb-b4dd-0faa84d4a64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.712575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 813/1618] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:00.714432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677fb7e-e2dc-4538-9548-9b5953dbcd66 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.720153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527909a6-79aa-44bb-b4dd-0faa84d4a64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.721637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527909a6-79aa-44bb-b4dd-0faa84d4a64b service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.730408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677fb7e-e2dc-4538-9548-9b5953dbcd66 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.734259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 806/1619] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:00.736809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8112b3-4a13-4db2-b3c6-a56873e54950 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.737199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8112b3-4a13-4db2-b3c6-a56873e54950 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.737639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8112b3-4a13-4db2-b3c6-a56873e54950 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.739956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 807/1620] 10.222.0.52 () {54 vars in 912 bytes} [Mon Oct 10 11:59:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:00.741030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8112b3-4a13-4db2-b3c6-a56873e54950 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.747227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3484559-2ed2-40a9-9b7c-0d24c22c539c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.750876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3484559-2ed2-40a9-9b7c-0d24c22c539c service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.751218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3484559-2ed2-40a9-9b7c-0d24c22c539c service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.751497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3484559-2ed2-40a9-9b7c-0d24c22c539c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.764368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8112b3-4a13-4db2-b3c6-a56873e54950 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.769812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3484559-2ed2-40a9-9b7c-0d24c22c539c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.771857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3484559-2ed2-40a9-9b7c-0d24c22c539c service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.777989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e2c68-e74e-4c35-b755-350debd6a837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.778665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e2c68-e74e-4c35-b755-350debd6a837 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.778983 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e2c68-e74e-4c35-b755-350debd6a837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.779253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e2c68-e74e-4c35-b755-350debd6a837 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.779882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e2c68-e74e-4c35-b755-350debd6a837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.782039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e2c68-e74e-4c35-b755-350debd6a837 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.794740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 814/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:00.798519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcb191-cea1-47a5-84aa-48b36d7561a3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.798817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcb191-cea1-47a5-84aa-48b36d7561a3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.799171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcb191-cea1-47a5-84aa-48b36d7561a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.830968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 808/1622] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:00 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:00.831933 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e2c68-e74e-4c35-b755-350debd6a837 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.840541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-16bcb191-cea1-47a5-84aa-48b36d7561a3 None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:00.841749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-16bcb191-cea1-47a5-84aa-48b36d7561a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:00.856306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e2c68-e74e-4c35-b755-350debd6a837 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.864821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451cc0f-fbf0-45ac-98c4-abf9de1c8f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.865509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451cc0f-fbf0-45ac-98c4-abf9de1c8f63 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.865794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451cc0f-fbf0-45ac-98c4-abf9de1c8f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.866066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451cc0f-fbf0-45ac-98c4-abf9de1c8f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.889370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451cc0f-fbf0-45ac-98c4-abf9de1c8f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.890906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451cc0f-fbf0-45ac-98c4-abf9de1c8f63 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.912762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 809/1623] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:00.913666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451cc0f-fbf0-45ac-98c4-abf9de1c8f63 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.940532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451cc0f-fbf0-45ac-98c4-abf9de1c8f63 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.954443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b180c-99d9-4bc4-885b-fb7c903d2786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:00.955326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b180c-99d9-4bc4-885b-fb7c903d2786 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:00.955458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b180c-99d9-4bc4-885b-fb7c903d2786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:00.955740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b180c-99d9-4bc4-885b-fb7c903d2786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:00.956577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b180c-99d9-4bc4-885b-fb7c903d2786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:00.958173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b180c-99d9-4bc4-885b-fb7c903d2786 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:00.973064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 810/1624] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:00.976902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b180c-99d9-4bc4-885b-fb7c903d2786 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:00.995776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b180c-99d9-4bc4-885b-fb7c903d2786 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:00.998210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 815/1625] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:01.002473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb52987-8c1a-457a-8409-127fb0bc77c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:01.003180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb52987-8c1a-457a-8409-127fb0bc77c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:01.003463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb52987-8c1a-457a-8409-127fb0bc77c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:01.003739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb52987-8c1a-457a-8409-127fb0bc77c6 admin admin] PATH_INFO: `/v3/projects/dc15daca44574bd88417810af03a4c2c/users/b6b81e8d9130478bbe281d7654e7c2de/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:01.012680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcb191-cea1-47a5-84aa-48b36d7561a3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:01.024339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb52987-8c1a-457a-8409-127fb0bc77c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc15daca44574bd88417810af03a4c2c, user_id=b6b81e8d9130478bbe281d7654e7c2de, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:01.028918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb52987-8c1a-457a-8409-127fb0bc77c6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:01.057843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 811/1626] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:00 2022] PUT /identity/v3/projects/dc15daca44574bd88417810af03a4c2c/users/b6b81e8d9130478bbe281d7654e7c2de/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:01.062960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb52987-8c1a-457a-8409-127fb0bc77c6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:01.077135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:01.099667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcb191-cea1-47a5-84aa-48b36d7561a3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:01.172433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b9a17-a7e8-45b3-a1c3-456e34d18a51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:01.173227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b9a17-a7e8-45b3-a1c3-456e34d18a51 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:01.173580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b9a17-a7e8-45b3-a1c3-456e34d18a51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:01.173864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b9a17-a7e8-45b3-a1c3-456e34d18a51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:01.228626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb52987-8c1a-457a-8409-127fb0bc77c6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:01.236190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527fa1b-c610-4603-b7ec-3d8088542151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:01.237439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527fa1b-c610-4603-b7ec-3d8088542151 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:01.237723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527fa1b-c610-4603-b7ec-3d8088542151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:01.238758 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527fa1b-c610-4603-b7ec-3d8088542151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:01.238758 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527fa1b-c610-4603-b7ec-3d8088542151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:01.241468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527fa1b-c610-4603-b7ec-3d8088542151 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:01.261378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 812/1627] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:01.264417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527fa1b-c610-4603-b7ec-3d8088542151 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:01.276925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527fa1b-c610-4603-b7ec-3d8088542151 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:01.289356 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201349-1afb-4631-9e33-791b258ee462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:01.289962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201349-1afb-4631-9e33-791b258ee462 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:01.290316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201349-1afb-4631-9e33-791b258ee462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:01.292085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201349-1afb-4631-9e33-791b258ee462 admin admin] PATH_INFO: `/v3/projects/dc15daca44574bd88417810af03a4c2c/users/b6b81e8d9130478bbe281d7654e7c2de/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:01.300248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d201349-1afb-4631-9e33-791b258ee462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc15daca44574bd88417810af03a4c2c, user_id=b6b81e8d9130478bbe281d7654e7c2de, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:01.310578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d201349-1afb-4631-9e33-791b258ee462 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:01.338414 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 813/1628] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:01 2022] PUT /identity/v3/projects/dc15daca44574bd88417810af03a4c2c/users/b6b81e8d9130478bbe281d7654e7c2de/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:01.356267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b9a17-a7e8-45b3-a1c3-456e34d18a51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:01.357130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8c5c-8b72-4950-ac21-9862d46bcd09 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:01.357456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8c5c-8b72-4950-ac21-9862d46bcd09 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:01.357704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8c5c-8b72-4950-ac21-9862d46bcd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:01.358764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b9a17-a7e8-45b3-a1c3-456e34d18a51 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:01.387186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 816/1629] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:01.409612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2fee8c5c-8b72-4950-ac21-9862d46bcd09 None None] MFA Rules not processed for user `b6b81e8d9130478bbe281d7654e7c2de`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:01.412018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2fee8c5c-8b72-4950-ac21-9862d46bcd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:01.550494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:01.618553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 814/1630] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:01.619270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b9a17-a7e8-45b3-a1c3-456e34d18a51 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:01.631111 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b9a17-a7e8-45b3-a1c3-456e34d18a51 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:01.808938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef86574-e60a-4358-adcd-83998c10a0e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:01.809833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86574-e60a-4358-adcd-83998c10a0e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:01.810279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86574-e60a-4358-adcd-83998c10a0e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:01.810708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86574-e60a-4358-adcd-83998c10a0e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:01.831412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef86574-e60a-4358-adcd-83998c10a0e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:01.833651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef86574-e60a-4358-adcd-83998c10a0e3 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:01.863278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 817/1631] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:59:02.138121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8c5c-8b72-4950-ac21-9862d46bcd09 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:02.156473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8c5c-8b72-4950-ac21-9862d46bcd09 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:02.167841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18aba6-23c8-43f6-b571-76989168441a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:02.168620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18aba6-23c8-43f6-b571-76989168441a service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:02.168909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18aba6-23c8-43f6-b571-76989168441a service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:02.170509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18aba6-23c8-43f6-b571-76989168441a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:02.341896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18aba6-23c8-43f6-b571-76989168441a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:02.348575 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18aba6-23c8-43f6-b571-76989168441a service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:02.385417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 815/1632] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:03.028117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625151c-ac16-4702-970c-2d1b2e5cc47d None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.028682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625151c-ac16-4702-970c-2d1b2e5cc47d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.028682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625151c-ac16-4702-970c-2d1b2e5cc47d None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.030245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 818/1633] 10.222.0.52 () {54 vars in 961 bytes} [Mon Oct 10 11:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:03.039803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69612512-2bb3-4ad7-bd49-b1403da6b391 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.040098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69612512-2bb3-4ad7-bd49-b1403da6b391 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.040360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69612512-2bb3-4ad7-bd49-b1403da6b391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.082469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-69612512-2bb3-4ad7-bd49-b1403da6b391 None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:03.084137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-69612512-2bb3-4ad7-bd49-b1403da6b391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:03.245928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 816/1634] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Oct 10 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:03.251941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87483d-3c23-4271-ba9f-8b9ecc09bd2d None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.252316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87483d-3c23-4271-ba9f-8b9ecc09bd2d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.252546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87483d-3c23-4271-ba9f-8b9ecc09bd2d None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.254186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 819/1635] 10.222.0.52 () {54 vars in 912 bytes} [Mon Oct 10 11:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:03.260445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477264d-99c7-462b-b9d7-63bb87283e2a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.261449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477264d-99c7-462b-b9d7-63bb87283e2a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.261896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477264d-99c7-462b-b9d7-63bb87283e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.323917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7477264d-99c7-462b-b9d7-63bb87283e2a None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:03.325192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7477264d-99c7-462b-b9d7-63bb87283e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:03.350913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 817/1636] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:03.366271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87483d-3c23-4271-ba9f-8b9ecc09bd2d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:03.383789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87483d-3c23-4271-ba9f-8b9ecc09bd2d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:03.397186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ee9e-5016-4323-b9fc-32dfa337760f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:03.398024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ee9e-5016-4323-b9fc-32dfa337760f service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.398410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ee9e-5016-4323-b9fc-32dfa337760f service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.398834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766ee9e-5016-4323-b9fc-32dfa337760f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.437178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766ee9e-5016-4323-b9fc-32dfa337760f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:03.438714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766ee9e-5016-4323-b9fc-32dfa337760f service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:03.472439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 820/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:05.096869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fe175-99ce-462c-8629-1c43123a19b2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:05.097480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fe175-99ce-462c-8629-1c43123a19b2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:05.097650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fe175-99ce-462c-8629-1c43123a19b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:05.131355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cf1fe175-99ce-462c-8629-1c43123a19b2 None None] MFA Rules not processed for user `e591a4012c0b4b7fa46dc1ef72958a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:05.133050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cf1fe175-99ce-462c-8629-1c43123a19b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:05.302501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 818/1638] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:05 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:05.307413 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ee9e-5016-4323-b9fc-32dfa337760f service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:05.388177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:05.523422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766ee9e-5016-4323-b9fc-32dfa337760f service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:05.552162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67299269-da7e-47f0-9025-eda0601bbced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:05.553805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299269-da7e-47f0-9025-eda0601bbced admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:05.553805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299269-da7e-47f0-9025-eda0601bbced admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:05.553805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299269-da7e-47f0-9025-eda0601bbced admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:05.558790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67299269-da7e-47f0-9025-eda0601bbced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:05.565893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67299269-da7e-47f0-9025-eda0601bbced admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:05.643859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 821/1639] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:05 2022] POST /identity/v3/projects => generated 403 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:05.644708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fe175-99ce-462c-8629-1c43123a19b2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:05.650740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:05.667334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fe175-99ce-462c-8629-1c43123a19b2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:05.841052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dcf4f-32fc-4d32-b2ce-f1fcd71e3fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:05.841690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcf4f-32fc-4d32-b2ce-f1fcd71e3fcc admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:05.841979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcf4f-32fc-4d32-b2ce-f1fcd71e3fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:05.842252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcf4f-32fc-4d32-b2ce-f1fcd71e3fcc admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:05.849018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dcf4f-32fc-4d32-b2ce-f1fcd71e3fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:05.857129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dcf4f-32fc-4d32-b2ce-f1fcd71e3fcc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:05.934978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 819/1640] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:05 2022] POST /identity/v3/users => generated 308 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:59:05.938716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67299269-da7e-47f0-9025-eda0601bbced admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:05.961516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67299269-da7e-47f0-9025-eda0601bbced admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:05.980982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537658b3-eba1-4ce3-803e-29801f0335ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:05.981660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537658b3-eba1-4ce3-803e-29801f0335ca admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:05.981935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537658b3-eba1-4ce3-803e-29801f0335ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:05.982204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537658b3-eba1-4ce3-803e-29801f0335ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:05.982956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537658b3-eba1-4ce3-803e-29801f0335ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:05.984578 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537658b3-eba1-4ce3-803e-29801f0335ca admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:06.019037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 822/1641] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:06.021476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dcf4f-32fc-4d32-b2ce-f1fcd71e3fcc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:06.047434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dcf4f-32fc-4d32-b2ce-f1fcd71e3fcc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:06.073704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35017dc-d1c7-4149-ba5d-0cdb69271e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:06.075073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35017dc-d1c7-4149-ba5d-0cdb69271e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.076581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35017dc-d1c7-4149-ba5d-0cdb69271e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.076872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35017dc-d1c7-4149-ba5d-0cdb69271e99 admin admin] PATH_INFO: `/v3/projects/c140b85800d2467483b359c52d92d217/users/092a69bca62945c8ae8f53e1984e0e88/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:06.110148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35017dc-d1c7-4149-ba5d-0cdb69271e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c140b85800d2467483b359c52d92d217, user_id=092a69bca62945c8ae8f53e1984e0e88, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:06.115178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35017dc-d1c7-4149-ba5d-0cdb69271e99 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:06.158632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 820/1642] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:06 2022] PUT /identity/v3/projects/c140b85800d2467483b359c52d92d217/users/092a69bca62945c8ae8f53e1984e0e88/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:06.159961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537658b3-eba1-4ce3-803e-29801f0335ca admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:06.172361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537658b3-eba1-4ce3-803e-29801f0335ca admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:06.260751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a942c0-5fad-410f-963a-c6055f5eeab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:06.261913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a942c0-5fad-410f-963a-c6055f5eeab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.262392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a942c0-5fad-410f-963a-c6055f5eeab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.262857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a942c0-5fad-410f-963a-c6055f5eeab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:06.263740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a942c0-5fad-410f-963a-c6055f5eeab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:06.265701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a942c0-5fad-410f-963a-c6055f5eeab2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:06.289011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595f541-0080-4de1-8906-dd5b8215eb5e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.289607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595f541-0080-4de1-8906-dd5b8215eb5e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.290113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595f541-0080-4de1-8906-dd5b8215eb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:06.312646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 823/1643] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:06.316909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a942c0-5fad-410f-963a-c6055f5eeab2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:06.330124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a942c0-5fad-410f-963a-c6055f5eeab2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:06.337823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf553d-f948-439c-a593-970691541ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:06.338873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf553d-f948-439c-a593-970691541ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.339391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf553d-f948-439c-a593-970691541ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.339800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf553d-f948-439c-a593-970691541ab0 admin admin] PATH_INFO: `/v3/projects/c140b85800d2467483b359c52d92d217/users/092a69bca62945c8ae8f53e1984e0e88/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:06.342319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf553d-f948-439c-a593-970691541ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c140b85800d2467483b359c52d92d217, user_id=092a69bca62945c8ae8f53e1984e0e88, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:06.346841 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf553d-f948-439c-a593-970691541ab0 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:06.350840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e595f541-0080-4de1-8906-dd5b8215eb5e None None] MFA Rules not processed for user `b051e414038b470691ee3e91b952a87d`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:06.352371 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e595f541-0080-4de1-8906-dd5b8215eb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:06.360564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 824/1644] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:06 2022] PUT /identity/v3/projects/c140b85800d2467483b359c52d92d217/users/092a69bca62945c8ae8f53e1984e0e88/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:59:06.389263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf553d-f948-439c-a593-970691541ab0 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:06.466672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf553d-f948-439c-a593-970691541ab0 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:06.472684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:06.473723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.474208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.474694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] PATH_INFO: `/v3/users/f633c812bb6446a49107a05b3c956366` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:06.476180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f633c812bb6446a49107a05b3c956366)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:06.477489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: WARNING py.warnings [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:59:06.477489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: warnings.warn(msg) Oct 10 11:59:06.477489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: Oct 10 11:59:06.480094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:06.505253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 821/1645] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:06.532223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595f541-0080-4de1-8906-dd5b8215eb5e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:06.536689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:06.595277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f633c812bb6446a49107a05b3c956366'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:06.612119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f633c812bb6446a49107a05b3c956366'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:06.613207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f633c812bb6446a49107a05b3c956366'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:06.613987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f633c812bb6446a49107a05b3c956366'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:06.637014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d3028734-fcee-4b41-8de8-9113f215500a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f633c812bb6446a49107a05b3c956366'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:06.648728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:06.684497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595f541-0080-4de1-8906-dd5b8215eb5e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:06.689568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 825/1646] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:06 2022] DELETE /identity/v3/users/f633c812bb6446a49107a05b3c956366 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:06.691872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117b94-0afc-4aaa-8238-2dce14c30c06 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.691872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117b94-0afc-4aaa-8238-2dce14c30c06 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.692680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117b94-0afc-4aaa-8238-2dce14c30c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:06.725005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a7117b94-0afc-4aaa-8238-2dce14c30c06 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:06.728840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a7117b94-0afc-4aaa-8238-2dce14c30c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:06.834360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:06.835430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.835718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.835993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:06.843217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:06.858772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 826/1647] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:06.874537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7117b94-0afc-4aaa-8238-2dce14c30c06 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:06.876724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a7117b94-0afc-4aaa-8238-2dce14c30c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:06.978958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:06.983727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:07.001015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7117b94-0afc-4aaa-8238-2dce14c30c06 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:07.008634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 822/1648] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:07.011421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b0c7a-ae9d-4a79-a87c-9b66bc96ecd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:07.011421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b0c7a-ae9d-4a79-a87c-9b66bc96ecd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:07.011421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b0c7a-ae9d-4a79-a87c-9b66bc96ecd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:07.011691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b0c7a-ae9d-4a79-a87c-9b66bc96ecd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:07.023143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b0c7a-ae9d-4a79-a87c-9b66bc96ecd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:07.026045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b0c7a-ae9d-4a79-a87c-9b66bc96ecd7 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:07.038147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 827/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:59:09.393410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:09.395569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:09.427505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007077ac-aff9-46d9-8fa7-d7a183587bdf service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:09.439750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:09.440624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:09.441277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:09.441710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] PATH_INFO: `/v3/projects/41c3718a446542109d01276a99950b84` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:09.454854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41c3718a446542109d01276a99950b84)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:09.462067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 11:59:09.462067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Oct 10 11:59:09.462067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: Oct 10 11:59:09.464286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:09.826797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41c3718a446542109d01276a99950b84', 'request_id': 'req-dd02250e-e2b5-4198-a2c5-a464bdee3742'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:09.862531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41c3718a446542109d01276a99950b84', 'request_id': 'req-dd02250e-e2b5-4198-a2c5-a464bdee3742'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:09.934979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41c3718a446542109d01276a99950b84', 'request_id': 'req-dd02250e-e2b5-4198-a2c5-a464bdee3742'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:09.968224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930875e6-71f0-44ac-b92e-79ae45ca9bed None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:09.969350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930875e6-71f0-44ac-b92e-79ae45ca9bed None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:09.969833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930875e6-71f0-44ac-b92e-79ae45ca9bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.006864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] The token cache is being invalidate because project 41c3718a446542109d01276a99950b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:10.007571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:10.008923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dd02250e-e2b5-4198-a2c5-a464bdee3742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:10.014827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 823/1650] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:09 2022] DELETE /identity/v3/projects/41c3718a446542109d01276a99950b84 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:10.020274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9b51-3b42-4001-9b6d-4ade07c50078 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.020777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9b51-3b42-4001-9b6d-4ade07c50078 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.021203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9b51-3b42-4001-9b6d-4ade07c50078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.054668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-930875e6-71f0-44ac-b92e-79ae45ca9bed None None] MFA Rules not processed for user `092a69bca62945c8ae8f53e1984e0e88`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:10.056214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-930875e6-71f0-44ac-b92e-79ae45ca9bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:10.069796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4a4e9b51-3b42-4001-9b6d-4ade07c50078 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:10.074328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4a4e9b51-3b42-4001-9b6d-4ade07c50078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:10.111986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:10.147803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:10.277864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 824/1651] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:10.281658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 828/1652] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:10.287155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123c00f-76f0-4156-8501-c484db402224 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.287155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123c00f-76f0-4156-8501-c484db402224 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.287155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123c00f-76f0-4156-8501-c484db402224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.334001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6123c00f-76f0-4156-8501-c484db402224 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:10.336433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6123c00f-76f0-4156-8501-c484db402224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:10.364404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 825/1653] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:10.369512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930875e6-71f0-44ac-b92e-79ae45ca9bed None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:10.382771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930875e6-71f0-44ac-b92e-79ae45ca9bed None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:10.389569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:10.390692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.391751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.392221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.393084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:59:10.393603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:10.395567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:10.432422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 829/1654] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:59:10.436747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123c00f-76f0-4156-8501-c484db402224 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:10.448689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123c00f-76f0-4156-8501-c484db402224 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:10.454205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9c275-c907-4fa7-9ef3-bd8e156f979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:10.455246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9c275-c907-4fa7-9ef3-bd8e156f979b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.456342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9c275-c907-4fa7-9ef3-bd8e156f979b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.456892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9c275-c907-4fa7-9ef3-bd8e156f979b admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.457784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9c275-c907-4fa7-9ef3-bd8e156f979b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:10.464206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9c275-c907-4fa7-9ef3-bd8e156f979b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:10.543266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 826/1655] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:10 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:10.547912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:10.566998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a2fd-e9d1-40a6-96e0-e8f9112ea7ca admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:10.718756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35553a-de79-426f-9bba-7c94ac26cd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:10.719451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35553a-de79-426f-9bba-7c94ac26cd34 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.719732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35553a-de79-426f-9bba-7c94ac26cd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.720055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35553a-de79-426f-9bba-7c94ac26cd34 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.720708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35553a-de79-426f-9bba-7c94ac26cd34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:10.722664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35553a-de79-426f-9bba-7c94ac26cd34 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:10.780728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9c275-c907-4fa7-9ef3-bd8e156f979b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:10.791320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 830/1656] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:10 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:10.797988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9c275-c907-4fa7-9ef3-bd8e156f979b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:10.813818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a50d3-56a6-4132-a1ec-eff2aa891c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:10.814468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a50d3-56a6-4132-a1ec-eff2aa891c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.814916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a50d3-56a6-4132-a1ec-eff2aa891c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.815515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a50d3-56a6-4132-a1ec-eff2aa891c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.816675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a50d3-56a6-4132-a1ec-eff2aa891c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:10.822741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a50d3-56a6-4132-a1ec-eff2aa891c59 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:10.848733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 827/1657] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:10.854832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35553a-de79-426f-9bba-7c94ac26cd34 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:10.873878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35553a-de79-426f-9bba-7c94ac26cd34 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:10.893310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aed799-a462-4830-8cff-db1b761b958b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:10.898513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed799-a462-4830-8cff-db1b761b958b admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.899920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed799-a462-4830-8cff-db1b761b958b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.901593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aed799-a462-4830-8cff-db1b761b958b admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.902275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aed799-a462-4830-8cff-db1b761b958b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:10.906692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aed799-a462-4830-8cff-db1b761b958b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:10.941009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a50d3-56a6-4132-a1ec-eff2aa891c59 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:10.942564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 831/1658] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:10.957424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a50d3-56a6-4132-a1ec-eff2aa891c59 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:10.976090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b9e78-6783-49b4-8dcc-cd0e2ce199ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:10.976781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9e78-6783-49b4-8dcc-cd0e2ce199ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.977643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9e78-6783-49b4-8dcc-cd0e2ce199ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.977643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9e78-6783-49b4-8dcc-cd0e2ce199ea admin admin] PATH_INFO: `/v3/projects/8c0d8f662c9c493db9529a6912881684/users/055e7bc4c72d4bf3b2f172d599757991/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:11.021345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b9e78-6783-49b4-8dcc-cd0e2ce199ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0d8f662c9c493db9529a6912881684, user_id=055e7bc4c72d4bf3b2f172d599757991, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:11.024363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b9e78-6783-49b4-8dcc-cd0e2ce199ea admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:11.077255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa17f-d08a-4692-9873-7a5fc527cb39 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:11.079001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa17f-d08a-4692-9873-7a5fc527cb39 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:11.079001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa17f-d08a-4692-9873-7a5fc527cb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:11.081763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 828/1659] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:10 2022] PUT /identity/v3/projects/8c0d8f662c9c493db9529a6912881684/users/055e7bc4c72d4bf3b2f172d599757991/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:11.117380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-adafa17f-d08a-4692-9873-7a5fc527cb39 None None] MFA Rules not processed for user `055e7bc4c72d4bf3b2f172d599757991`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:11.120807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-adafa17f-d08a-4692-9873-7a5fc527cb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:11.282549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 832/1660] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:11.289562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b9e78-6783-49b4-8dcc-cd0e2ce199ea admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:11.475696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b9e78-6783-49b4-8dcc-cd0e2ce199ea admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:11.490591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3cf68-1b93-4198-90ac-d24e7922568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:11.491576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3cf68-1b93-4198-90ac-d24e7922568c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:11.492007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3cf68-1b93-4198-90ac-d24e7922568c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:11.492480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3cf68-1b93-4198-90ac-d24e7922568c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:11.495192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3cf68-1b93-4198-90ac-d24e7922568c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:11.500940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3cf68-1b93-4198-90ac-d24e7922568c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:11.584127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 829/1661] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:11 2022] POST /identity/v3/projects => generated 373 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:11.584792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafa17f-d08a-4692-9873-7a5fc527cb39 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:11.604273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafa17f-d08a-4692-9873-7a5fc527cb39 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:11.642512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:11.734071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c47fd-b606-4f5f-8ecf-e342cdfb5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:11.734705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c47fd-b606-4f5f-8ecf-e342cdfb5600 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:11.735141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c47fd-b606-4f5f-8ecf-e342cdfb5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:11.735413 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c47fd-b606-4f5f-8ecf-e342cdfb5600 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:11.736060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c47fd-b606-4f5f-8ecf-e342cdfb5600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:11.748561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c47fd-b606-4f5f-8ecf-e342cdfb5600 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:11.823100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 833/1662] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:11 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:11.828936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3cf68-1b93-4198-90ac-d24e7922568c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:11.856155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3cf68-1b93-4198-90ac-d24e7922568c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:11.873192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f8414-50c2-407d-be4c-4ae84de08533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:11.873845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8414-50c2-407d-be4c-4ae84de08533 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:11.874123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8414-50c2-407d-be4c-4ae84de08533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:11.874391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8414-50c2-407d-be4c-4ae84de08533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:11.875081 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f8414-50c2-407d-be4c-4ae84de08533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:11.881225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f8414-50c2-407d-be4c-4ae84de08533 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:11.905039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 830/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:11.906263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c47fd-b606-4f5f-8ecf-e342cdfb5600 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:11.922000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c47fd-b606-4f5f-8ecf-e342cdfb5600 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:11.954393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0df8d-24cd-4228-8747-639a6435ecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:11.964223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0df8d-24cd-4228-8747-639a6435ecc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:11.964658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0df8d-24cd-4228-8747-639a6435ecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:11.965010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0df8d-24cd-4228-8747-639a6435ecc1 admin admin] PATH_INFO: `/v3/projects/1d7ccbbbc8e54ca58f168e342062bb56/users/ee56bc99373a4e29ad0376135861f008/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:12.054033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0df8d-24cd-4228-8747-639a6435ecc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7ccbbbc8e54ca58f168e342062bb56, user_id=ee56bc99373a4e29ad0376135861f008, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:12.055522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0df8d-24cd-4228-8747-639a6435ecc1 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:12.088474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 834/1664] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:11 2022] PUT /identity/v3/projects/1d7ccbbbc8e54ca58f168e342062bb56/users/ee56bc99373a4e29ad0376135861f008/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:12.093396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f8414-50c2-407d-be4c-4ae84de08533 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:12.111053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f8414-50c2-407d-be4c-4ae84de08533 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:12.196021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:12.258222 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ccd8b-96f0-4184-b997-73adab58b31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:12.258863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ccd8b-96f0-4184-b997-73adab58b31c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:12.259242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ccd8b-96f0-4184-b997-73adab58b31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:12.259570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ccd8b-96f0-4184-b997-73adab58b31c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:12.260807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ccd8b-96f0-4184-b997-73adab58b31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:12.263464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ccd8b-96f0-4184-b997-73adab58b31c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:12.276610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 831/1665] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:12.280431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0df8d-24cd-4228-8747-639a6435ecc1 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:12.296645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0df8d-24cd-4228-8747-639a6435ecc1 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:12.315929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:12.317041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:12.318315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:12.318828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] PATH_INFO: `/v3/projects/1d7ccbbbc8e54ca58f168e342062bb56/users/ee56bc99373a4e29ad0376135861f008/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:12.321459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7ccbbbc8e54ca58f168e342062bb56, user_id=ee56bc99373a4e29ad0376135861f008, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:12.325467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:12.347208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 835/1666] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:12 2022] PUT /identity/v3/projects/1d7ccbbbc8e54ca58f168e342062bb56/users/ee56bc99373a4e29ad0376135861f008/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:12.358699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563254c2-b163-47b4-99c9-8c6760cbd4f5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:12.359071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563254c2-b163-47b4-99c9-8c6760cbd4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:12.359349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563254c2-b163-47b4-99c9-8c6760cbd4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:12.428977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-563254c2-b163-47b4-99c9-8c6760cbd4f5 None None] MFA Rules not processed for user `ee56bc99373a4e29ad0376135861f008`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:12.431212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-563254c2-b163-47b4-99c9-8c6760cbd4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:12.665732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 832/1667] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:12.676203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:12.678019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:12.802427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7336694-b54b-42d4-8c4e-f505efc8976c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:12.807079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f7203-f2a4-4cb2-b2b1-7709babdcde4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:12.807766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f7203-f2a4-4cb2-b2b1-7709babdcde4 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:12.808108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f7203-f2a4-4cb2-b2b1-7709babdcde4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:12.808387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f7203-f2a4-4cb2-b2b1-7709babdcde4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:12.826483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f7203-f2a4-4cb2-b2b1-7709babdcde4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:12.828003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f7203-f2a4-4cb2-b2b1-7709babdcde4 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:12.857062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 836/1668] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:12.995512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563254c2-b163-47b4-99c9-8c6760cbd4f5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:13.015962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1627b23-616c-4fb6-9412-fc513b5b3b7d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:13.016990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1627b23-616c-4fb6-9412-fc513b5b3b7d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:13.018015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1627b23-616c-4fb6-9412-fc513b5b3b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:13.044313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563254c2-b163-47b4-99c9-8c6760cbd4f5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:13.059191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03790b8-7d79-4791-bb26-b93cef1e1865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:13.059902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03790b8-7d79-4791-bb26-b93cef1e1865 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:13.061518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03790b8-7d79-4791-bb26-b93cef1e1865 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:13.061806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03790b8-7d79-4791-bb26-b93cef1e1865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:13.075321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e1627b23-616c-4fb6-9412-fc513b5b3b7d None None] MFA Rules not processed for user `b051e414038b470691ee3e91b952a87d`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:13.080034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e1627b23-616c-4fb6-9412-fc513b5b3b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:13.285581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03790b8-7d79-4791-bb26-b93cef1e1865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:13.301923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03790b8-7d79-4791-bb26-b93cef1e1865 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:13.302139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 837/1669] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:13.335162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 833/1670] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:13.589131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452612d-35ba-4e0e-8d5a-f498188c3cf2 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:13.589907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452612d-35ba-4e0e-8d5a-f498188c3cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:13.589907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452612d-35ba-4e0e-8d5a-f498188c3cf2 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:13.592249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 838/1671] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 11:59:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:13.742163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb93583-4648-4e87-979b-e24a747fa53c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:13.742619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb93583-4648-4e87-979b-e24a747fa53c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:13.742710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb93583-4648-4e87-979b-e24a747fa53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:13.794978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cfb93583-4648-4e87-979b-e24a747fa53c None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:13.797046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cfb93583-4648-4e87-979b-e24a747fa53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:13.917262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:13.977340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 834/1672] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 11:59:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:14.202112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452612d-35ba-4e0e-8d5a-f498188c3cf2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:14.204328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a452612d-35ba-4e0e-8d5a-f498188c3cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:14.239672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452612d-35ba-4e0e-8d5a-f498188c3cf2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:14.265109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:14.266768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:14.267085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:14.267405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:14.288055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:14.290134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:14.308565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 839/1673] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:59:14.394986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835cd0f-44c8-4b92-b29a-848826de3fd4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:14.395281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835cd0f-44c8-4b92-b29a-848826de3fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:14.395772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835cd0f-44c8-4b92-b29a-848826de3fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:14.447473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a835cd0f-44c8-4b92-b29a-848826de3fd4 None None] MFA Rules not processed for user `2f7735444ca8431ea7906070374dbe67`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:14.448709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a835cd0f-44c8-4b92-b29a-848826de3fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:14.583644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 835/1674] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:14.596351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:14.597674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:14.621854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:14.769307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e25ac-ac61-4ef7-9097-a9f47c3c9f0e service placement] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:14.781356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a165438-27f3-4781-98ce-eede5c5840a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:14.784523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a165438-27f3-4781-98ce-eede5c5840a1 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:14.785041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a165438-27f3-4781-98ce-eede5c5840a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:14.785729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a165438-27f3-4781-98ce-eede5c5840a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:14.805106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a165438-27f3-4781-98ce-eede5c5840a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:14.807670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a165438-27f3-4781-98ce-eede5c5840a1 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:14.843888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 840/1675] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:16.075892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eca6af-3576-4757-80d0-4004fa8c1e3a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:16.076732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eca6af-3576-4757-80d0-4004fa8c1e3a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:16.077868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eca6af-3576-4757-80d0-4004fa8c1e3a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:16.086189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 836/1676] 10.222.0.52 () {54 vars in 912 bytes} [Mon Oct 10 11:59:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:16.093620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b99d1-325e-4317-b635-d904b383c3ce None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:16.094574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b99d1-325e-4317-b635-d904b383c3ce None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:16.094759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b99d1-325e-4317-b635-d904b383c3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:16.183131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c96b99d1-325e-4317-b635-d904b383c3ce None None] MFA Rules not processed for user `48ad70aa047d443b97e13447397a8a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:16.185593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c96b99d1-325e-4317-b635-d904b383c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:16.268988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 841/1677] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 11:59:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:16.293034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eca6af-3576-4757-80d0-4004fa8c1e3a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:16.296494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-92eca6af-3576-4757-80d0-4004fa8c1e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:16.327235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eca6af-3576-4757-80d0-4004fa8c1e3a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:16.341214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2621a39-f964-484a-a2b8-ac3de924d766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:16.342623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2621a39-f964-484a-a2b8-ac3de924d766 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:16.343278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2621a39-f964-484a-a2b8-ac3de924d766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:16.343852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2621a39-f964-484a-a2b8-ac3de924d766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:16.378441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2621a39-f964-484a-a2b8-ac3de924d766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:16.386906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2621a39-f964-484a-a2b8-ac3de924d766 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:16.424053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 837/1678] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:17.388965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b99d1-325e-4317-b635-d904b383c3ce None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:17.390716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c96b99d1-325e-4317-b635-d904b383c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:17.639737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b99d1-325e-4317-b635-d904b383c3ce None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:17.653227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:17.653897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:17.654176 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:17.654447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] PATH_INFO: `/v3/users/e591a4012c0b4b7fa46dc1ef72958a68` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:17.657400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e591a4012c0b4b7fa46dc1ef72958a68)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:17.667433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:17.691280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2621a39-f964-484a-a2b8-ac3de924d766 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:17.739231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2621a39-f964-484a-a2b8-ac3de924d766 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:17.761190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:17.764195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:17.764745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:17.765278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:17.768134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:17.783597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:17.786566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:17.788438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:17.817954 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 838/1679] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:59:17.841975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e591a4012c0b4b7fa46dc1ef72958a68'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:17.871456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e591a4012c0b4b7fa46dc1ef72958a68'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:17.878579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e591a4012c0b4b7fa46dc1ef72958a68'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:17.886664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e591a4012c0b4b7fa46dc1ef72958a68'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:17.922924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e591a4012c0b4b7fa46dc1ef72958a68'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:17.979089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 842/1680] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:17 2022] DELETE /identity/v3/users/e591a4012c0b4b7fa46dc1ef72958a68 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:19.853684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:19.873372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fabd2c-7ea7-4489-b047-3716c6e821f4 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:20.031380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:20.032504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:20.032901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:20.033992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] PATH_INFO: `/v3/projects/830b3f1b187e4811a3180a5f07490691` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:20.035413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=830b3f1b187e4811a3180a5f07490691)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:20.041945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:20.153295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:20.161330 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830b3f1b187e4811a3180a5f07490691', 'request_id': 'req-3029e513-49b2-48ff-b477-28ac58f09c7e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.182251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830b3f1b187e4811a3180a5f07490691', 'request_id': 'req-3029e513-49b2-48ff-b477-28ac58f09c7e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.203813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830b3f1b187e4811a3180a5f07490691', 'request_id': 'req-3029e513-49b2-48ff-b477-28ac58f09c7e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.220819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] The token cache is being invalidate because project 830b3f1b187e4811a3180a5f07490691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:20.221495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.222488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3029e513-49b2-48ff-b477-28ac58f09c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.225489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 839/1681] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:19 2022] DELETE /identity/v3/projects/830b3f1b187e4811a3180a5f07490691 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:20.229437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:20.231555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:20.249346 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b4f52-7371-4445-af2e-69e460ad948f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:20.517197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:20.518173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:20.518656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:20.519143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] PATH_INFO: `/v3/users/092a69bca62945c8ae8f53e1984e0e88` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:20.520561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] RBAC: Authorizing `identity:delete_user(user_id=092a69bca62945c8ae8f53e1984e0e88)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:20.522745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:20.714672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '092a69bca62945c8ae8f53e1984e0e88'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.741242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '092a69bca62945c8ae8f53e1984e0e88'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.742039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '092a69bca62945c8ae8f53e1984e0e88'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.743914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '092a69bca62945c8ae8f53e1984e0e88'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.766796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fc21829-cbc3-41ce-bab5-5c83d98f603c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '092a69bca62945c8ae8f53e1984e0e88'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:20.811351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 843/1682] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:20 2022] DELETE /identity/v3/users/092a69bca62945c8ae8f53e1984e0e88 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:20.846475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd68dfa-6f9e-4494-9668-7aa1dd4e8421 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:20.847150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd68dfa-6f9e-4494-9668-7aa1dd4e8421 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:20.847820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd68dfa-6f9e-4494-9668-7aa1dd4e8421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:20.928359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6bd68dfa-6f9e-4494-9668-7aa1dd4e8421 None None] MFA Rules not processed for user `0c0f1d116c004694b33b19d988ac4959`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:20.933301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6bd68dfa-6f9e-4494-9668-7aa1dd4e8421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:21.008052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84447ce-4801-45e1-92ca-24f4bd814234 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:21.008346 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84447ce-4801-45e1-92ca-24f4bd814234 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:21.008640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84447ce-4801-45e1-92ca-24f4bd814234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:21.064062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b84447ce-4801-45e1-92ca-24f4bd814234 None None] MFA Rules not processed for user `2f7735444ca8431ea7906070374dbe67`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:21.065316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b84447ce-4801-45e1-92ca-24f4bd814234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:21.135756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 840/1683] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:21.156174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:21.159465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd68dfa-6f9e-4494-9668-7aa1dd4e8421 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:21.162811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6bd68dfa-6f9e-4494-9668-7aa1dd4e8421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:21.239859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 844/1684] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:21.373344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd68dfa-6f9e-4494-9668-7aa1dd4e8421 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:21.391554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508b521-d244-4e96-8ebf-91e041955dd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:21.392423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508b521-d244-4e96-8ebf-91e041955dd1 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:21.392912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508b521-d244-4e96-8ebf-91e041955dd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:21.393483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508b521-d244-4e96-8ebf-91e041955dd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:21.429087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508b521-d244-4e96-8ebf-91e041955dd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:21.431183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508b521-d244-4e96-8ebf-91e041955dd1 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:21.471972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 841/1685] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:21.496848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84447ce-4801-45e1-92ca-24f4bd814234 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:21.499203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b84447ce-4801-45e1-92ca-24f4bd814234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:21.674931 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84447ce-4801-45e1-92ca-24f4bd814234 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:21.686708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:21.691317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:21.691615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:21.691900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] PATH_INFO: `/v3/projects/c140b85800d2467483b359c52d92d217` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:21.693199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c140b85800d2467483b359c52d92d217)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:21.700945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:21.810640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c140b85800d2467483b359c52d92d217', 'request_id': 'req-587a9402-0b66-45ce-bce0-bdd7bce0ef04'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:21.827085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c140b85800d2467483b359c52d92d217', 'request_id': 'req-587a9402-0b66-45ce-bce0-bdd7bce0ef04'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:21.848780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c140b85800d2467483b359c52d92d217', 'request_id': 'req-587a9402-0b66-45ce-bce0-bdd7bce0ef04'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:21.886824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] The token cache is being invalidate because project c140b85800d2467483b359c52d92d217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:21.889502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:21.892897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-587a9402-0b66-45ce-bce0-bdd7bce0ef04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:21.896698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 845/1686] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:21 2022] DELETE /identity/v3/projects/c140b85800d2467483b359c52d92d217 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:21.904576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a745a5-6840-4984-8455-3d1449ca5721 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:21.904850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a745a5-6840-4984-8455-3d1449ca5721 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:21.905133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a745a5-6840-4984-8455-3d1449ca5721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.005124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-01a745a5-6840-4984-8455-3d1449ca5721 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:22.007691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-01a745a5-6840-4984-8455-3d1449ca5721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:22.128374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:22.164667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b19dfb-612b-4d8e-99c6-1ea12fcfa676 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.165863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b19dfb-612b-4d8e-99c6-1ea12fcfa676 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.166441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b19dfb-612b-4d8e-99c6-1ea12fcfa676 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.168389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 846/1687] 10.222.0.52 () {54 vars in 912 bytes} [Mon Oct 10 11:59:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:59:22.173392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6354-bab9-4844-85df-945036505ddd None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.173797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6354-bab9-4844-85df-945036505ddd None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.174179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6354-bab9-4844-85df-945036505ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.179958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 842/1688] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:22.185858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f048-759f-4af1-b649-9286b2214ac6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.186306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f048-759f-4af1-b649-9286b2214ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.186730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f048-759f-4af1-b649-9286b2214ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.245046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-08fa6354-bab9-4844-85df-945036505ddd None None] MFA Rules not processed for user `48ad70aa047d443b97e13447397a8a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:22.247173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-08fa6354-bab9-4844-85df-945036505ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:22.253721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2a42f048-759f-4af1-b649-9286b2214ac6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:22.258466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2a42f048-759f-4af1-b649-9286b2214ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:22.283199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 843/1689] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:22.284648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42f048-759f-4af1-b649-9286b2214ac6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:22.303722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42f048-759f-4af1-b649-9286b2214ac6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:22.319486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca1712-d737-4918-a114-015455533f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:22.321478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1712-d737-4918-a114-015455533f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.323618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1712-d737-4918-a114-015455533f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.323618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1712-d737-4918-a114-015455533f50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.324911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca1712-d737-4918-a114-015455533f50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:59:22.326434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca1712-d737-4918-a114-015455533f50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:22.330490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca1712-d737-4918-a114-015455533f50 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:22.367733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 844/1690] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:59:22.369105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca1712-d737-4918-a114-015455533f50 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:22.373077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 847/1691] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:22.398197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca1712-d737-4918-a114-015455533f50 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:22.411322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ed994-b1c8-4e39-9463-709ad251dc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:22.417376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed994-b1c8-4e39-9463-709ad251dc6d admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.417673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed994-b1c8-4e39-9463-709ad251dc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.417956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed994-b1c8-4e39-9463-709ad251dc6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.418608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ed994-b1c8-4e39-9463-709ad251dc6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:22.420699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ed994-b1c8-4e39-9463-709ad251dc6d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:22.459513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 845/1692] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:22 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:22.464011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa6354-bab9-4844-85df-945036505ddd None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:22.496513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa6354-bab9-4844-85df-945036505ddd None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:22.513008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:22.594652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914b7c-3662-47fd-bbe6-8a266b24c913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:22.595480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914b7c-3662-47fd-bbe6-8a266b24c913 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.595859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914b7c-3662-47fd-bbe6-8a266b24c913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.597877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3914b7c-3662-47fd-bbe6-8a266b24c913 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.598946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3914b7c-3662-47fd-bbe6-8a266b24c913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:22.601579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3914b7c-3662-47fd-bbe6-8a266b24c913 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:22.658122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 848/1693] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:22 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:59:22.659505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ed994-b1c8-4e39-9463-709ad251dc6d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:22.676711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ed994-b1c8-4e39-9463-709ad251dc6d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:22.688819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3dd7b-f63e-404d-9b21-64007ecd8b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:22.689536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3dd7b-f63e-404d-9b21-64007ecd8b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.689868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3dd7b-f63e-404d-9b21-64007ecd8b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.690158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3dd7b-f63e-404d-9b21-64007ecd8b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.691583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3dd7b-f63e-404d-9b21-64007ecd8b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:22.697586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3dd7b-f63e-404d-9b21-64007ecd8b41 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:22.722600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 846/1694] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:22.723959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914b7c-3662-47fd-bbe6-8a266b24c913 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:22.742398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3914b7c-3662-47fd-bbe6-8a266b24c913 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:22.761374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff72e-bc30-4dec-a261-05b8ce70927d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.761677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff72e-bc30-4dec-a261-05b8ce70927d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.761959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff72e-bc30-4dec-a261-05b8ce70927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.762482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:22.764513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.766856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.767388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.768407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:22.773401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:22.805324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 849/1695] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:59:22.810522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:22.819057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:22.883079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-878ff72e-bc30-4dec-a261-05b8ce70927d None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:22.886460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-878ff72e-bc30-4dec-a261-05b8ce70927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:22.976425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf88994-678a-4953-a441-3b204df4383e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:22.987727 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5dcce-00b4-48b8-bf79-0aa571f66b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:22.988470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5dcce-00b4-48b8-bf79-0aa571f66b41 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.988888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5dcce-00b4-48b8-bf79-0aa571f66b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.989415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5dcce-00b4-48b8-bf79-0aa571f66b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:23.022098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5dcce-00b4-48b8-bf79-0aa571f66b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:23.025546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5dcce-00b4-48b8-bf79-0aa571f66b41 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:23.048271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 850/1696] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:23.050148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5dcce-00b4-48b8-bf79-0aa571f66b41 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:23.052271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 847/1697] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:23.053803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8928a2d-3eca-4104-86e9-c3dd1d21c392 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:23.054109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8928a2d-3eca-4104-86e9-c3dd1d21c392 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:23.054650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8928a2d-3eca-4104-86e9-c3dd1d21c392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:23.116138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d8928a2d-3eca-4104-86e9-c3dd1d21c392 None None] MFA Rules not processed for user `8568f2501f824875bae7ab1825cc2c32`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:23.117564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d8928a2d-3eca-4104-86e9-c3dd1d21c392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:23.154015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5dcce-00b4-48b8-bf79-0aa571f66b41 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:23.160759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:23.162480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:23.162972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:23.164596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] PATH_INFO: `/v3/projects/917e1c692458462093d7c7fa7b48d3a4/users/cc2d4bb115dc48e4a782e2a04a3b68f5/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:23.197262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] RBAC: Authorizing `identity:create_grant(project_id=917e1c692458462093d7c7fa7b48d3a4, user_id=cc2d4bb115dc48e4a782e2a04a3b68f5, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:23.201209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:23.252390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 851/1698] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:23 2022] PUT /identity/v3/projects/917e1c692458462093d7c7fa7b48d3a4/users/cc2d4bb115dc48e4a782e2a04a3b68f5/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:23.253255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:23.255571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:23.282171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:23.302058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 848/1699] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:23.308588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c6ca-5807-4ff0-9801-c9c54c074aa6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:23.308882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c6ca-5807-4ff0-9801-c9c54c074aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:23.309163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c6ca-5807-4ff0-9801-c9c54c074aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:23.375395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e674c6ca-5807-4ff0-9801-c9c54c074aa6 None None] MFA Rules not processed for user `cc2d4bb115dc48e4a782e2a04a3b68f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:23.377835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e674c6ca-5807-4ff0-9801-c9c54c074aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:23.383883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fb139-dda4-4b7f-a81f-10f7e9d1b56b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:23.395708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:23.396554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:23.396628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:23.398944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] PATH_INFO: `/v3/users/55d073e4d9af4c7f923889350ba74cda` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:23.400972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d073e4d9af4c7f923889350ba74cda)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:23.402936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:23.447311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:23.520232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d073e4d9af4c7f923889350ba74cda'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:23.538828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d073e4d9af4c7f923889350ba74cda'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:23.540861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d073e4d9af4c7f923889350ba74cda'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:23.543758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d073e4d9af4c7f923889350ba74cda'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:23.555000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 849/1700] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:23.556341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314df329-bac0-4e7b-b111-a3090e0c368f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:23.558644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314df329-bac0-4e7b-b111-a3090e0c368f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:23.559028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314df329-bac0-4e7b-b111-a3090e0c368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:23.562780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d073e4d9af4c7f923889350ba74cda'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:23.624202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 852/1701] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:23 2022] DELETE /identity/v3/users/55d073e4d9af4c7f923889350ba74cda => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:23.624202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:23.625944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:23.626431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-314df329-bac0-4e7b-b111-a3090e0c368f None None] MFA Rules not processed for user `8568f2501f824875bae7ab1825cc2c32`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:23.630030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-314df329-bac0-4e7b-b111-a3090e0c368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:23.790812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 850/1702] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:23.791685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314df329-bac0-4e7b-b111-a3090e0c368f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:23.796153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-314df329-bac0-4e7b-b111-a3090e0c368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:23.837649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f40b8-1997-4968-bbba-51a537886d4f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:23.851246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e49517-46ee-4724-8ca4-65c91cd3ec13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:23.851957 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49517-46ee-4724-8ca4-65c91cd3ec13 service swift] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:23.852282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49517-46ee-4724-8ca4-65c91cd3ec13 service swift] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:23.853200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49517-46ee-4724-8ca4-65c91cd3ec13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:23.856009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-09e49517-46ee-4724-8ca4-65c91cd3ec13 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:23.963893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314df329-bac0-4e7b-b111-a3090e0c368f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:23.978728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3e742-ae3a-4303-a2b1-3f8ad7f83d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:23.982484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e742-ae3a-4303-a2b1-3f8ad7f83d01 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:23.982484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e742-ae3a-4303-a2b1-3f8ad7f83d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:23.983253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e742-ae3a-4303-a2b1-3f8ad7f83d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:23.983927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3e742-ae3a-4303-a2b1-3f8ad7f83d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:23.989407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3e742-ae3a-4303-a2b1-3f8ad7f83d01 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:24.055096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e49517-46ee-4724-8ca4-65c91cd3ec13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:24.058664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e49517-46ee-4724-8ca4-65c91cd3ec13 service swift] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:24.063095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 851/1703] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:23 2022] POST /identity/v3/projects => generated 381 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:24.064898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c55e29-752e-4ed2-87fa-1f5112558b88 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:24.065166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c55e29-752e-4ed2-87fa-1f5112558b88 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:24.065499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c55e29-752e-4ed2-87fa-1f5112558b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:24.079415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 853/1704] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 11:59:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 11:59:24.083397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdf028-b63b-4090-9023-8b4bec9b22e7 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:24.083686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdf028-b63b-4090-9023-8b4bec9b22e7 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:24.083962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdf028-b63b-4090-9023-8b4bec9b22e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:24.095251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-35c55e29-752e-4ed2-87fa-1f5112558b88 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:24.096565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-35c55e29-752e-4ed2-87fa-1f5112558b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:24.116276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:24.127565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:24.147983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b1bdf028-b63b-4090-9023-8b4bec9b22e7 None None] MFA Rules not processed for user `99b3e727e6fa480c90a054a60b68b8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:24.150510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b1bdf028-b63b-4090-9023-8b4bec9b22e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:24.276941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 852/1705] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:24.277946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c55e29-752e-4ed2-87fa-1f5112558b88 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:24.282301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-35c55e29-752e-4ed2-87fa-1f5112558b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:24.312742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 854/1706] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:24.318760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdf028-b63b-4090-9023-8b4bec9b22e7 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:24.350784 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdf028-b63b-4090-9023-8b4bec9b22e7 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:24.363407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:24.364318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:24.364752 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:24.365177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:24.367553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:24.373362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:24.462287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 855/1707] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:24 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:24.486881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:24.498838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:24.511376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c55e29-752e-4ed2-87fa-1f5112558b88 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:24.527366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d1fb1-b361-4135-b626-3270a82b4023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:24.527781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d1fb1-b361-4135-b626-3270a82b4023 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:24.528380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d1fb1-b361-4135-b626-3270a82b4023 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:24.528659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d1fb1-b361-4135-b626-3270a82b4023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:24.530532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e62d1fb1-b361-4135-b626-3270a82b4023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:24.875207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ffb00-2a3b-472f-b26b-217b868c9c34 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:24.890376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df26f6-ba85-4f12-9026-79583e0e7aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:24.896783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df26f6-ba85-4f12-9026-79583e0e7aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:24.897762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df26f6-ba85-4f12-9026-79583e0e7aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:24.898499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df26f6-ba85-4f12-9026-79583e0e7aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:24.923235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d1fb1-b361-4135-b626-3270a82b4023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:24.929777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d1fb1-b361-4135-b626-3270a82b4023 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:24.940024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df26f6-ba85-4f12-9026-79583e0e7aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:24.957749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df26f6-ba85-4f12-9026-79583e0e7aa5 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:24.991161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 853/1708] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:24.998930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63213a-2a32-459c-abdf-45f4be495494 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:24.998930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63213a-2a32-459c-abdf-45f4be495494 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:24.998930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63213a-2a32-459c-abdf-45f4be495494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.004189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 856/1709] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:25.006938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df26f6-ba85-4f12-9026-79583e0e7aa5 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:25.039348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df26f6-ba85-4f12-9026-79583e0e7aa5 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:25.069830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b52572-01d9-42f2-9017-699dcf5abb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:25.070522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b52572-01d9-42f2-9017-699dcf5abb32 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:25.070806 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b52572-01d9-42f2-9017-699dcf5abb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:25.076071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b52572-01d9-42f2-9017-699dcf5abb32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.078174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b52572-01d9-42f2-9017-699dcf5abb32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:25.082011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6a63213a-2a32-459c-abdf-45f4be495494 None None] MFA Rules not processed for user `99b3e727e6fa480c90a054a60b68b8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:25.085467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b52572-01d9-42f2-9017-699dcf5abb32 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:25.087397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6a63213a-2a32-459c-abdf-45f4be495494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:25.124518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 854/1710] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:25.130833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e0771-3a76-4dfc-9a12-e7e020ac2faa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:25.136367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e0771-3a76-4dfc-9a12-e7e020ac2faa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:25.137649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e0771-3a76-4dfc-9a12-e7e020ac2faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.139593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 857/1711] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:25.147102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b52572-01d9-42f2-9017-699dcf5abb32 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:25.231189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b52572-01d9-42f2-9017-699dcf5abb32 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:25.256830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6433-a8e6-4864-acd3-e3b932746d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:25.257545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6433-a8e6-4864-acd3-e3b932746d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:25.257810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6433-a8e6-4864-acd3-e3b932746d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:25.261245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6433-a8e6-4864-acd3-e3b932746d59 admin admin] PATH_INFO: `/v3/projects/8b4864272377416ca29b0c0660e3f8c3/users/585d3eba10624289821811c7d57e8baa/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.264301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d45e0771-3a76-4dfc-9a12-e7e020ac2faa None None] MFA Rules not processed for user `48ad70aa047d443b97e13447397a8a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:25.272450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d45e0771-3a76-4dfc-9a12-e7e020ac2faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:25.300976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 855/1712] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Oct 10 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:25.320058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e0771-3a76-4dfc-9a12-e7e020ac2faa None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:25.322552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d45e0771-3a76-4dfc-9a12-e7e020ac2faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:25.330188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be6433-a8e6-4864-acd3-e3b932746d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4864272377416ca29b0c0660e3f8c3, user_id=585d3eba10624289821811c7d57e8baa, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:25.341433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be6433-a8e6-4864-acd3-e3b932746d59 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:25.358877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e0771-3a76-4dfc-9a12-e7e020ac2faa None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:25.394521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 858/1713] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:25 2022] PUT /identity/v3/projects/8b4864272377416ca29b0c0660e3f8c3/users/585d3eba10624289821811c7d57e8baa/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:25.407051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6433-a8e6-4864-acd3-e3b932746d59 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:25.639493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:25.678646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4e790-3294-46a8-97a6-5e8bc8e6736f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:25.679837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4e790-3294-46a8-97a6-5e8bc8e6736f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:25.681295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4e790-3294-46a8-97a6-5e8bc8e6736f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:25.683247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4e790-3294-46a8-97a6-5e8bc8e6736f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.691375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:25.704594 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4e790-3294-46a8-97a6-5e8bc8e6736f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:25.706093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4e790-3294-46a8-97a6-5e8bc8e6736f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:25.733848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6433-a8e6-4864-acd3-e3b932746d59 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:25.739749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 856/1714] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:59:25.747379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dd706-13ea-4a92-8ed4-4fcef6f80e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:25.747619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dd706-13ea-4a92-8ed4-4fcef6f80e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:25.748219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dd706-13ea-4a92-8ed4-4fcef6f80e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:25.748335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dd706-13ea-4a92-8ed4-4fcef6f80e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.754845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dd706-13ea-4a92-8ed4-4fcef6f80e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:25.761552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dd706-13ea-4a92-8ed4-4fcef6f80e90 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:25.837925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4e790-3294-46a8-97a6-5e8bc8e6736f service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:25.840809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 859/1715] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:25.854925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4e790-3294-46a8-97a6-5e8bc8e6736f service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:25.860678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:25.861347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:25.861623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:25.861950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] PATH_INFO: `/v3/projects/8b4864272377416ca29b0c0660e3f8c3/users/585d3eba10624289821811c7d57e8baa/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.867985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4864272377416ca29b0c0660e3f8c3, user_id=585d3eba10624289821811c7d57e8baa, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:25.871029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:25.904425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196f39c-5aff-4bfc-9652-64ff1cee5e7b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:25.905017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196f39c-5aff-4bfc-9652-64ff1cee5e7b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:25.905600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196f39c-5aff-4bfc-9652-64ff1cee5e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.910310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 857/1716] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:25 2022] PUT /identity/v3/projects/8b4864272377416ca29b0c0660e3f8c3/users/585d3eba10624289821811c7d57e8baa/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:26.003792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a196f39c-5aff-4bfc-9652-64ff1cee5e7b None None] MFA Rules not processed for user `585d3eba10624289821811c7d57e8baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:26.020793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a196f39c-5aff-4bfc-9652-64ff1cee5e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:26.286058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 860/1717] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:26.297692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:26.300487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:26.494435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ba2e0-e2dc-4d2f-82b9-2e70e5657cd8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:26.509349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:26.510385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:26.514645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:26.515283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:26.542967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:26.547474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:26.582464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 858/1718] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:27.218944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196f39c-5aff-4bfc-9652-64ff1cee5e7b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:27.228278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a196f39c-5aff-4bfc-9652-64ff1cee5e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:27.537432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196f39c-5aff-4bfc-9652-64ff1cee5e7b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:27.563962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bde15-f93d-4067-9394-23b46f1d721c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:27.563962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bde15-f93d-4067-9394-23b46f1d721c service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:27.563962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bde15-f93d-4067-9394-23b46f1d721c service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:27.566442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bde15-f93d-4067-9394-23b46f1d721c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:27.790770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bde15-f93d-4067-9394-23b46f1d721c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:27.793000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bde15-f93d-4067-9394-23b46f1d721c service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:27.838606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 861/1719] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:28.072229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:28.080833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bde15-f93d-4067-9394-23b46f1d721c service nova] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:28.083563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:28.132067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:28.298694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bde15-f93d-4067-9394-23b46f1d721c service nova] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:28.313413 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:28.314350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:28.314851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:28.319602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] PATH_INFO: `/v3/users/cc2d4bb115dc48e4a782e2a04a3b68f5` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:28.324886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2d4bb115dc48e4a782e2a04a3b68f5)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:28.327468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:28.408000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea845e2-2b6a-47de-ab86-876d4eaad0bc service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:28.423600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:28.425367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:28.425747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:28.425917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] PATH_INFO: `/v3/projects/4293c37dca19479297f1e60de20a865f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:28.428128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4293c37dca19479297f1e60de20a865f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:28.434052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:28.506016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2d4bb115dc48e4a782e2a04a3b68f5'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.564093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4293c37dca19479297f1e60de20a865f', 'request_id': 'req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.576237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4293c37dca19479297f1e60de20a865f', 'request_id': 'req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.585770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4293c37dca19479297f1e60de20a865f', 'request_id': 'req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.597238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2d4bb115dc48e4a782e2a04a3b68f5'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.600593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2d4bb115dc48e4a782e2a04a3b68f5'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.602166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2d4bb115dc48e4a782e2a04a3b68f5'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.622673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1a8df017-a3ee-40cf-a47f-f0737be5f851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2d4bb115dc48e4a782e2a04a3b68f5'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.627013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] The token cache is being invalidate because project 4293c37dca19479297f1e60de20a865f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:28.627327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.637659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:28.645976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 859/1720] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:28 2022] DELETE /identity/v3/projects/4293c37dca19479297f1e60de20a865f => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:28.653461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:28.667394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:28.693772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 862/1721] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:28 2022] DELETE /identity/v3/users/cc2d4bb115dc48e4a782e2a04a3b68f5 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:28.695329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7bcc2-4245-4ec9-bee7-98e09389ee15 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:28.695848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7bcc2-4245-4ec9-bee7-98e09389ee15 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:28.696491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7bcc2-4245-4ec9-bee7-98e09389ee15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:28.732452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:28.751216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ecb7bcc2-4245-4ec9-bee7-98e09389ee15 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:28.757906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ecb7bcc2-4245-4ec9-bee7-98e09389ee15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:28.952319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 863/1722] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:28.978055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7bcc2-4245-4ec9-bee7-98e09389ee15 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:28.980270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ecb7bcc2-4245-4ec9-bee7-98e09389ee15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:29.074330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:29.133985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7bcc2-4245-4ec9-bee7-98e09389ee15 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:29.147616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c9219-e767-4111-9a4e-0714569f057f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:29.148314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c9219-e767-4111-9a4e-0714569f057f service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:29.148590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c9219-e767-4111-9a4e-0714569f057f service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:29.148907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c9219-e767-4111-9a4e-0714569f057f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:29.159804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ea51-2df3-4273-8fd2-f848b33b2f07 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:29.164553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c9219-e767-4111-9a4e-0714569f057f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:29.166072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c9219-e767-4111-9a4e-0714569f057f service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:29.194253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 864/1723] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:29.195557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:29.200277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:29.200666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:29.200850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] PATH_INFO: `/v3/users/b6b81e8d9130478bbe281d7654e7c2de` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:29.208919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b81e8d9130478bbe281d7654e7c2de)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:29.210951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:29.361410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e70a0-ebc5-403a-b722-31c9693bca39 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:29.362424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e70a0-ebc5-403a-b722-31c9693bca39 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:29.362993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e70a0-ebc5-403a-b722-31c9693bca39 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:29.364651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 865/1724] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 11:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:59:29.445279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b81e8d9130478bbe281d7654e7c2de'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:29.487212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b81e8d9130478bbe281d7654e7c2de'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:29.500043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b81e8d9130478bbe281d7654e7c2de'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:29.502146 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b81e8d9130478bbe281d7654e7c2de'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:29.510846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b2b7c-556f-4895-bf52-b483d9250ad4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:29.511180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b2b7c-556f-4895-bf52-b483d9250ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:29.511618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b2b7c-556f-4895-bf52-b483d9250ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:29.559933 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b81e8d9130478bbe281d7654e7c2de'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:29.597052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-313b2b7c-556f-4895-bf52-b483d9250ad4 None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:29.598291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-313b2b7c-556f-4895-bf52-b483d9250ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:29.642542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 860/1725] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:28 2022] DELETE /identity/v3/users/b6b81e8d9130478bbe281d7654e7c2de => generated 0 bytes in 993 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:29.884461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 866/1726] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 11:59:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:31.934852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:31.937127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:32.220177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970038b0-c2df-487f-aa05-4ae90dbcffff admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:32.243546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5de72-0971-48b3-8d7c-53d0f704077f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:32.245246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5de72-0971-48b3-8d7c-53d0f704077f service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:32.245534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5de72-0971-48b3-8d7c-53d0f704077f service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:32.245810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5de72-0971-48b3-8d7c-53d0f704077f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:32.291076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5de72-0971-48b3-8d7c-53d0f704077f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:32.292685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5de72-0971-48b3-8d7c-53d0f704077f service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:32.341487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 861/1727] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:32.387466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b2b7c-556f-4895-bf52-b483d9250ad4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:32.388944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-313b2b7c-556f-4895-bf52-b483d9250ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:32.436124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b2b7c-556f-4895-bf52-b483d9250ad4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:32.741773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:32.743526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:32.744090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:32.745372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] PATH_INFO: `/v3/projects/dc15daca44574bd88417810af03a4c2c` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:32.753885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc15daca44574bd88417810af03a4c2c)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:32.788591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:32.990866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc15daca44574bd88417810af03a4c2c', 'request_id': 'req-57d12f37-0849-48c4-99e2-c07b2075a7a7'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:33.028589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc15daca44574bd88417810af03a4c2c', 'request_id': 'req-57d12f37-0849-48c4-99e2-c07b2075a7a7'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:33.038448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc15daca44574bd88417810af03a4c2c', 'request_id': 'req-57d12f37-0849-48c4-99e2-c07b2075a7a7'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:33.052797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:33.069909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] The token cache is being invalidate because project dc15daca44574bd88417810af03a4c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:33.070129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:33.071732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-57d12f37-0849-48c4-99e2-c07b2075a7a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:33.078735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 867/1728] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:32 2022] DELETE /identity/v3/projects/dc15daca44574bd88417810af03a4c2c => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:33.083717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff02a-fc66-4f3d-89a6-4b9188509441 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:33.084306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff02a-fc66-4f3d-89a6-4b9188509441 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:33.084558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ff02a-fc66-4f3d-89a6-4b9188509441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:33.139937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d07ff02a-fc66-4f3d-89a6-4b9188509441 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:33.141536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d07ff02a-fc66-4f3d-89a6-4b9188509441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:33.167785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:33.271155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099267dc-765f-4fb5-81ea-9350225ea62c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:33.271922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099267dc-765f-4fb5-81ea-9350225ea62c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:33.271922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099267dc-765f-4fb5-81ea-9350225ea62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:33.343571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-099267dc-765f-4fb5-81ea-9350225ea62c None None] MFA Rules not processed for user `0c0f1d116c004694b33b19d988ac4959`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:33.345801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-099267dc-765f-4fb5-81ea-9350225ea62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:33.357010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 862/1729] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:33.365347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d62a0-0b7a-41f3-9fc3-d09261ab6366 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:33.367046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d62a0-0b7a-41f3-9fc3-d09261ab6366 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:33.367548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d62a0-0b7a-41f3-9fc3-d09261ab6366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:33.457959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 868/1730] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:33.472900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9e3d62a0-0b7a-41f3-9fc3-d09261ab6366 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:33.474339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9e3d62a0-0b7a-41f3-9fc3-d09261ab6366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:33.495258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 863/1731] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:33.499707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099267dc-765f-4fb5-81ea-9350225ea62c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:33.530120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099267dc-765f-4fb5-81ea-9350225ea62c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:33.555925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:33.557449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:33.557809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:33.559967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:33.560743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:59:33.561126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:33.566144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:33.611311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 869/1732] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:59:33.613158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d62a0-0b7a-41f3-9fc3-d09261ab6366 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:33.629713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d62a0-0b7a-41f3-9fc3-d09261ab6366 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:33.668604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dd802-e76b-4b0d-925f-849e50a16dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:33.670086 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd802-e76b-4b0d-925f-849e50a16dbe admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:33.670603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd802-e76b-4b0d-925f-849e50a16dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:33.671033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dd802-e76b-4b0d-925f-849e50a16dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:33.671836 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dd802-e76b-4b0d-925f-849e50a16dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:33.677254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dd802-e76b-4b0d-925f-849e50a16dbe admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:33.775585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 864/1733] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:33 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:33.777126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:33.798717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d21eb-2f47-4c8a-84e1-18159a6670ad admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:33.932304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f714b-32b5-41d7-8097-1ccc5b69c7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:33.934858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f714b-32b5-41d7-8097-1ccc5b69c7cf admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:33.936077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f714b-32b5-41d7-8097-1ccc5b69c7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:33.937283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f714b-32b5-41d7-8097-1ccc5b69c7cf admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:33.938858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f714b-32b5-41d7-8097-1ccc5b69c7cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:33.956164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f714b-32b5-41d7-8097-1ccc5b69c7cf admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:34.007418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 870/1734] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:33 2022] POST /identity/v3/users => generated 291 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:34.011402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dd802-e76b-4b0d-925f-849e50a16dbe admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:34.047473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dd802-e76b-4b0d-925f-849e50a16dbe admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:34.068642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9104e4d-2cd2-45c9-ba5a-5f4a9136b2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:34.069287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104e4d-2cd2-45c9-ba5a-5f4a9136b2dd admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:34.069565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104e4d-2cd2-45c9-ba5a-5f4a9136b2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:34.069833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104e4d-2cd2-45c9-ba5a-5f4a9136b2dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:34.070488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9104e4d-2cd2-45c9-ba5a-5f4a9136b2dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:34.074495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9104e4d-2cd2-45c9-ba5a-5f4a9136b2dd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:34.129178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f714b-32b5-41d7-8097-1ccc5b69c7cf admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:34.129659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 865/1735] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:34.143322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f714b-32b5-41d7-8097-1ccc5b69c7cf admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:34.160912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd96e8-48d8-433a-a478-b87b8b36fb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:34.161969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd96e8-48d8-433a-a478-b87b8b36fb80 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:34.162174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd96e8-48d8-433a-a478-b87b8b36fb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:34.162609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd96e8-48d8-433a-a478-b87b8b36fb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:34.163374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd96e8-48d8-433a-a478-b87b8b36fb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:34.166395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd96e8-48d8-433a-a478-b87b8b36fb80 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:34.208029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 871/1736] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:34.212874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9104e4d-2cd2-45c9-ba5a-5f4a9136b2dd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:34.240117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9104e4d-2cd2-45c9-ba5a-5f4a9136b2dd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:34.259328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7b285-5f7c-42f3-a2f7-c8da03a1cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:34.260237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7b285-5f7c-42f3-a2f7-c8da03a1cb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:34.260681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7b285-5f7c-42f3-a2f7-c8da03a1cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:34.261148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7b285-5f7c-42f3-a2f7-c8da03a1cb0a admin admin] PATH_INFO: `/v3/projects/d3b16432751f4cd28120819f2c2b6701/users/fbc54579fd9c4aaca358097207c1980d/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:34.315803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7b285-5f7c-42f3-a2f7-c8da03a1cb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3b16432751f4cd28120819f2c2b6701, user_id=fbc54579fd9c4aaca358097207c1980d, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:34.320878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7b285-5f7c-42f3-a2f7-c8da03a1cb0a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:34.375682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 866/1737] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:34 2022] PUT /identity/v3/projects/d3b16432751f4cd28120819f2c2b6701/users/fbc54579fd9c4aaca358097207c1980d/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:34.387331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10046ab6-d609-43d6-98f4-17ac83c531c9 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:34.387878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10046ab6-d609-43d6-98f4-17ac83c531c9 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:34.388418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10046ab6-d609-43d6-98f4-17ac83c531c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:34.493733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-10046ab6-d609-43d6-98f4-17ac83c531c9 None None] MFA Rules not processed for user `fbc54579fd9c4aaca358097207c1980d`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:34.501480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-10046ab6-d609-43d6-98f4-17ac83c531c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:34.721881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:34.753303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 872/1738] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:34.754335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7b285-5f7c-42f3-a2f7-c8da03a1cb0a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:34.975898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10046ab6-d609-43d6-98f4-17ac83c531c9 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:34.978268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-10046ab6-d609-43d6-98f4-17ac83c531c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:35.011689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7b285-5f7c-42f3-a2f7-c8da03a1cb0a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:35.023123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01360a6-d65e-4857-acb4-bc54e3ca7be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:35.024676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01360a6-d65e-4857-acb4-bc54e3ca7be9 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.025098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01360a6-d65e-4857-acb4-bc54e3ca7be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.025510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01360a6-d65e-4857-acb4-bc54e3ca7be9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.026349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01360a6-d65e-4857-acb4-bc54e3ca7be9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:35.029390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01360a6-d65e-4857-acb4-bc54e3ca7be9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:35.071664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 867/1739] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:34 2022] POST /identity/v3/projects => generated 381 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:35.076761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01360a6-d65e-4857-acb4-bc54e3ca7be9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:35.191343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:35.217928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10046ab6-d609-43d6-98f4-17ac83c531c9 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:35.317394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01360a6-d65e-4857-acb4-bc54e3ca7be9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:35.338866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98971cc4-fc19-450a-9ae5-aad6e97e108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:35.339520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971cc4-fc19-450a-9ae5-aad6e97e108f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.339787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971cc4-fc19-450a-9ae5-aad6e97e108f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.340058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971cc4-fc19-450a-9ae5-aad6e97e108f admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.340695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98971cc4-fc19-450a-9ae5-aad6e97e108f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:35.345706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98971cc4-fc19-450a-9ae5-aad6e97e108f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:35.432101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 868/1740] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:35 2022] POST /identity/v3/users => generated 297 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:35.434641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98971cc4-fc19-450a-9ae5-aad6e97e108f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:35.472521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98971cc4-fc19-450a-9ae5-aad6e97e108f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:35.489017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce59b8-4374-4525-ad0d-17e65575816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:35.492068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce59b8-4374-4525-ad0d-17e65575816f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.492422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce59b8-4374-4525-ad0d-17e65575816f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.493786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce59b8-4374-4525-ad0d-17e65575816f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.494460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce59b8-4374-4525-ad0d-17e65575816f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:35.495506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:35.497356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.498035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.498509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce59b8-4374-4525-ad0d-17e65575816f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:35.499660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] PATH_INFO: `/v3/projects/917e1c692458462093d7c7fa7b48d3a4` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.505209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=917e1c692458462093d7c7fa7b48d3a4)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:35.511602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:35.563153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 869/1741] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:35.566389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce59b8-4374-4525-ad0d-17e65575816f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:35.602392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce59b8-4374-4525-ad0d-17e65575816f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:35.634814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa9c1c-2e07-4273-907c-d245c3a7482d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:35.635608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa9c1c-2e07-4273-907c-d245c3a7482d admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.636144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa9c1c-2e07-4273-907c-d245c3a7482d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.636144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa9c1c-2e07-4273-907c-d245c3a7482d admin admin] PATH_INFO: `/v3/projects/a7d515641c6748939e8967adcceea34c/users/c34e75ba624c4915a039c80f9c9a22e3/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.718616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa9c1c-2e07-4273-907c-d245c3a7482d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d515641c6748939e8967adcceea34c, user_id=c34e75ba624c4915a039c80f9c9a22e3, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:35.719932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917e1c692458462093d7c7fa7b48d3a4', 'request_id': 'req-ee60d58f-dcd3-4568-83b1-cf264d1b1843'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:35.735507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa9c1c-2e07-4273-907c-d245c3a7482d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:35.740794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917e1c692458462093d7c7fa7b48d3a4', 'request_id': 'req-ee60d58f-dcd3-4568-83b1-cf264d1b1843'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:35.766756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917e1c692458462093d7c7fa7b48d3a4', 'request_id': 'req-ee60d58f-dcd3-4568-83b1-cf264d1b1843'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:35.784678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 870/1742] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:35 2022] PUT /identity/v3/projects/a7d515641c6748939e8967adcceea34c/users/c34e75ba624c4915a039c80f9c9a22e3/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:35.791580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dfb21-9daf-4e9b-83f0-e2ddce667c82 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.795778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dfb21-9daf-4e9b-83f0-e2ddce667c82 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.797161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dfb21-9daf-4e9b-83f0-e2ddce667c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.832291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] The token cache is being invalidate because project 917e1c692458462093d7c7fa7b48d3a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:35.834993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:35.838007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:35.842169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 873/1743] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:34 2022] DELETE /identity/v3/projects/917e1c692458462093d7c7fa7b48d3a4 => generated 0 bytes in 867 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:35.857559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:35.862088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:35.882353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8b9dfb21-9daf-4e9b-83f0-e2ddce667c82 None None] MFA Rules not processed for user `99b3e727e6fa480c90a054a60b68b8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:35.891992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8b9dfb21-9daf-4e9b-83f0-e2ddce667c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:36.038777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 871/1744] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:36.040478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dfb21-9daf-4e9b-83f0-e2ddce667c82 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:36.042753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8b9dfb21-9daf-4e9b-83f0-e2ddce667c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:36.048557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:36.141167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60d58f-dcd3-4568-83b1-cf264d1b1843 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:36.171451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dfb21-9daf-4e9b-83f0-e2ddce667c82 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:36.179797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:36.183449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:36.184016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:36.184998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] PATH_INFO: `/v3/users/585d3eba10624289821811c7d57e8baa` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:36.188942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] RBAC: Authorizing `identity:delete_user(user_id=585d3eba10624289821811c7d57e8baa)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:36.204267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:36.237930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca45b0-bbeb-47f1-ab8e-3ce9f040f0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:36.238883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca45b0-bbeb-47f1-ab8e-3ce9f040f0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:36.239356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca45b0-bbeb-47f1-ab8e-3ce9f040f0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:36.239773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca45b0-bbeb-47f1-ab8e-3ce9f040f0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:36.240921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca45b0-bbeb-47f1-ab8e-3ce9f040f0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:36.244612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca45b0-bbeb-47f1-ab8e-3ce9f040f0e5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:36.258864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 874/1745] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:36.266883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca45b0-bbeb-47f1-ab8e-3ce9f040f0e5 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:36.298332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca45b0-bbeb-47f1-ab8e-3ce9f040f0e5 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:36.314087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8920e8-bb08-4c92-8093-66ae0e9512a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:36.315376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8920e8-bb08-4c92-8093-66ae0e9512a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:36.315666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8920e8-bb08-4c92-8093-66ae0e9512a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:36.315940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8920e8-bb08-4c92-8093-66ae0e9512a3 admin admin] PATH_INFO: `/v3/projects/a7d515641c6748939e8967adcceea34c/users/c34e75ba624c4915a039c80f9c9a22e3/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:36.318832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8920e8-bb08-4c92-8093-66ae0e9512a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d515641c6748939e8967adcceea34c, user_id=c34e75ba624c4915a039c80f9c9a22e3, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:36.323659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8920e8-bb08-4c92-8093-66ae0e9512a3 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:36.339512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:36.359260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 875/1746] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:36 2022] PUT /identity/v3/projects/a7d515641c6748939e8967adcceea34c/users/c34e75ba624c4915a039c80f9c9a22e3/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:36.370182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '585d3eba10624289821811c7d57e8baa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:36.372526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503519e7-f460-479f-a103-f68c1a46d4b2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:36.372957 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503519e7-f460-479f-a103-f68c1a46d4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:36.373323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503519e7-f460-479f-a103-f68c1a46d4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:36.374855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '585d3eba10624289821811c7d57e8baa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:36.376504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '585d3eba10624289821811c7d57e8baa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:36.377578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '585d3eba10624289821811c7d57e8baa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:36.394709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '585d3eba10624289821811c7d57e8baa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:36.448821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 872/1747] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:36 2022] DELETE /identity/v3/users/585d3eba10624289821811c7d57e8baa => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:36.463206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-503519e7-f460-479f-a103-f68c1a46d4b2 None None] MFA Rules not processed for user `c34e75ba624c4915a039c80f9c9a22e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:36.464640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-503519e7-f460-479f-a103-f68c1a46d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:36.793565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 876/1748] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:36.806331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:36.809109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:37.078875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a46ed6-333e-40b7-8a0c-475e6f9f284d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:37.093926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:37.096376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:37.096615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:37.096885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:37.119999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:37.122958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:37.170677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 873/1749] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:37.397239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503519e7-f460-479f-a103-f68c1a46d4b2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:37.415446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503519e7-f460-479f-a103-f68c1a46d4b2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:37.434733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723ecc5-e7a3-4af5-96f5-89f0e4699e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:37.436239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ecc5-e7a3-4af5-96f5-89f0e4699e72 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:37.438885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ecc5-e7a3-4af5-96f5-89f0e4699e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:37.439599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ecc5-e7a3-4af5-96f5-89f0e4699e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:37.450426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d723ecc5-e7a3-4af5-96f5-89f0e4699e72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:37.600988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:37.619993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723ecc5-e7a3-4af5-96f5-89f0e4699e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:37.621540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723ecc5-e7a3-4af5-96f5-89f0e4699e72 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:37.635104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 877/1750] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:38.867193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:38.869213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:38.989457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa081-b46f-418e-9cc8-9ecb61b4a727 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:39.006044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:39.006861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:39.007296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:39.007663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] PATH_INFO: `/v3/projects/8b4864272377416ca29b0c0660e3f8c3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:39.012452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4864272377416ca29b0c0660e3f8c3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:39.015267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:39.141447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4864272377416ca29b0c0660e3f8c3', 'request_id': 'req-863186f9-597f-4a81-8006-31f86e638139'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:39.168114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4864272377416ca29b0c0660e3f8c3', 'request_id': 'req-863186f9-597f-4a81-8006-31f86e638139'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:39.169864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:39.186951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4864272377416ca29b0c0660e3f8c3', 'request_id': 'req-863186f9-597f-4a81-8006-31f86e638139'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:39.214901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] The token cache is being invalidate because project 8b4864272377416ca29b0c0660e3f8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:39.215218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:39.218803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-863186f9-597f-4a81-8006-31f86e638139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:39.221300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 874/1751] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:38 2022] DELETE /identity/v3/projects/8b4864272377416ca29b0c0660e3f8c3 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:39.228792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03381d44-fc54-4d2e-9783-81393082eec4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:39.229716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03381d44-fc54-4d2e-9783-81393082eec4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:39.230212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03381d44-fc54-4d2e-9783-81393082eec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:39.262226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-03381d44-fc54-4d2e-9783-81393082eec4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:39.266697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-03381d44-fc54-4d2e-9783-81393082eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:39.375702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 878/1752] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:39.377195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f910ee-52fd-4de6-801f-0cfb83795c98 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:39.377484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f910ee-52fd-4de6-801f-0cfb83795c98 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:39.377766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f910ee-52fd-4de6-801f-0cfb83795c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:39.438369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a7f910ee-52fd-4de6-801f-0cfb83795c98 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:39.440317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a7f910ee-52fd-4de6-801f-0cfb83795c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:39.459467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 875/1753] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:39.463236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03381d44-fc54-4d2e-9783-81393082eec4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:39.475349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03381d44-fc54-4d2e-9783-81393082eec4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:39.487445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:39.488342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:39.489448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:39.491275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:39.492119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:59:39.494121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:39.495995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:39.551985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 879/1754] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:59:39.557297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f910ee-52fd-4de6-801f-0cfb83795c98 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:39.581110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f910ee-52fd-4de6-801f-0cfb83795c98 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:39.593824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955c522-c6a6-4ee2-a607-ad5a06336c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:39.595847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955c522-c6a6-4ee2-a607-ad5a06336c0c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:39.611174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955c522-c6a6-4ee2-a607-ad5a06336c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:39.611174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955c522-c6a6-4ee2-a607-ad5a06336c0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:39.611174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955c522-c6a6-4ee2-a607-ad5a06336c0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:39.612897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955c522-c6a6-4ee2-a607-ad5a06336c0c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:39.705655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 876/1755] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:39 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:39.716434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:39.732889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2450ca0-6d37-4f66-8afd-209905bbe761 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:40.040104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2745a3-fa15-43ac-bbf1-9ae98db41939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:40.040805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2745a3-fa15-43ac-bbf1-9ae98db41939 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:40.041087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2745a3-fa15-43ac-bbf1-9ae98db41939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:40.041356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2745a3-fa15-43ac-bbf1-9ae98db41939 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:40.041991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2745a3-fa15-43ac-bbf1-9ae98db41939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:40.047621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2745a3-fa15-43ac-bbf1-9ae98db41939 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:40.147874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955c522-c6a6-4ee2-a607-ad5a06336c0c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:40.148470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 880/1756] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:39 2022] POST /identity/v3/users => generated 288 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:40.168896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955c522-c6a6-4ee2-a607-ad5a06336c0c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:40.194744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a21366-46f1-400e-a490-614587eee520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:40.201725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a21366-46f1-400e-a490-614587eee520 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:40.203665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a21366-46f1-400e-a490-614587eee520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:40.204126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a21366-46f1-400e-a490-614587eee520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:40.206378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a21366-46f1-400e-a490-614587eee520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:40.214340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a21366-46f1-400e-a490-614587eee520 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:40.271529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 877/1757] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:59:40.275715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2745a3-fa15-43ac-bbf1-9ae98db41939 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:40.291309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2745a3-fa15-43ac-bbf1-9ae98db41939 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:40.307924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb08246-390d-46d9-bd51-c0240c4f788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:40.308892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08246-390d-46d9-bd51-c0240c4f788e admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:40.309292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08246-390d-46d9-bd51-c0240c4f788e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:40.309741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb08246-390d-46d9-bd51-c0240c4f788e admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:40.310940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb08246-390d-46d9-bd51-c0240c4f788e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:40.313185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb08246-390d-46d9-bd51-c0240c4f788e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:40.346749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a21366-46f1-400e-a490-614587eee520 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:40.348730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 881/1758] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:40.375460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a21366-46f1-400e-a490-614587eee520 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:40.390527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7e3b9-02a7-4337-89f8-8da40408304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:40.405560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7e3b9-02a7-4337-89f8-8da40408304f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:40.408099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7e3b9-02a7-4337-89f8-8da40408304f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:40.408407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7e3b9-02a7-4337-89f8-8da40408304f admin admin] PATH_INFO: `/v3/projects/a9de0fc88a2e45a4b2e5eb091f113848/users/9ee6c56817f4402ca489954ebfda89d3/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:40.515428 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7e3b9-02a7-4337-89f8-8da40408304f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9de0fc88a2e45a4b2e5eb091f113848, user_id=9ee6c56817f4402ca489954ebfda89d3, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:40.521603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7e3b9-02a7-4337-89f8-8da40408304f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:40.551587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 878/1759] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:40 2022] PUT /identity/v3/projects/a9de0fc88a2e45a4b2e5eb091f113848/users/9ee6c56817f4402ca489954ebfda89d3/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:59:40.561994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f6b3-d219-4d30-9cbd-65cba66aab12 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:40.562934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f6b3-d219-4d30-9cbd-65cba66aab12 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:40.563466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f6b3-d219-4d30-9cbd-65cba66aab12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:40.631050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-72c6f6b3-d219-4d30-9cbd-65cba66aab12 None None] MFA Rules not processed for user `9ee6c56817f4402ca489954ebfda89d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:40.637693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-72c6f6b3-d219-4d30-9cbd-65cba66aab12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:40.780264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:40.844336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 882/1760] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:40.848549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7e3b9-02a7-4337-89f8-8da40408304f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.015857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7e3b9-02a7-4337-89f8-8da40408304f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.035157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.035983 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.036366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.036763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.037563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.039753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.079425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 879/1761] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:40 2022] POST /identity/v3/projects => generated 373 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:41.083538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6f6b3-d219-4d30-9cbd-65cba66aab12 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.097354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6f6b3-d219-4d30-9cbd-65cba66aab12 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.162306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.164065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.177620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a04c-c99d-4cb9-865c-5be7bd629474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.178264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a04c-c99d-4cb9-865c-5be7bd629474 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.178550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a04c-c99d-4cb9-865c-5be7bd629474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.178822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a04c-c99d-4cb9-865c-5be7bd629474 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.179776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a04c-c99d-4cb9-865c-5be7bd629474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.182279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a04c-c99d-4cb9-865c-5be7bd629474 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.207101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 883/1762] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:41 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:41.208162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a04c-c99d-4cb9-865c-5be7bd629474 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.217952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a04c-c99d-4cb9-865c-5be7bd629474 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.222938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279196c-bf77-4a25-aa86-d0c95b41b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.223530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279196c-bf77-4a25-aa86-d0c95b41b2ba admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.223814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279196c-bf77-4a25-aa86-d0c95b41b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.224088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279196c-bf77-4a25-aa86-d0c95b41b2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.224723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279196c-bf77-4a25-aa86-d0c95b41b2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.226401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279196c-bf77-4a25-aa86-d0c95b41b2ba admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.234923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5370-a34b-4daf-ba67-0cbfceaee6fc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.237819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 884/1763] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:41.241497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279196c-bf77-4a25-aa86-d0c95b41b2ba admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.243676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.244327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.244606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.244877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] PATH_INFO: `/v3/users/fbc54579fd9c4aaca358097207c1980d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.246037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc54579fd9c4aaca358097207c1980d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.248057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.253674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279196c-bf77-4a25-aa86-d0c95b41b2ba admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.254121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:41.260990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7f8fc-1891-4bab-b525-4ad649397367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.263015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7f8fc-1891-4bab-b525-4ad649397367 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.263302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7f8fc-1891-4bab-b525-4ad649397367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.263867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7f8fc-1891-4bab-b525-4ad649397367 admin admin] PATH_INFO: `/v3/projects/130cc2faca61482fb6651e1ca3266b97/users/7071faea27344fac88a74f91af96b6f1/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.285242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7f8fc-1891-4bab-b525-4ad649397367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=130cc2faca61482fb6651e1ca3266b97, user_id=7071faea27344fac88a74f91af96b6f1, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.287992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7f8fc-1891-4bab-b525-4ad649397367 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.302085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 885/1764] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:41 2022] PUT /identity/v3/projects/130cc2faca61482fb6651e1ca3266b97/users/7071faea27344fac88a74f91af96b6f1/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:59:41.304908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc54579fd9c4aaca358097207c1980d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:41.307039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7f8fc-1891-4bab-b525-4ad649397367 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.312489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc54579fd9c4aaca358097207c1980d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:41.313255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc54579fd9c4aaca358097207c1980d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:41.313970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc54579fd9c4aaca358097207c1980d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:41.321710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0fc274da-d69a-42d4-9da4-162d63da0430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc54579fd9c4aaca358097207c1980d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:41.342644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 880/1765] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:41 2022] DELETE /identity/v3/users/fbc54579fd9c4aaca358097207c1980d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:59:41.346941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46addd-e9ed-4533-bc7a-6d3ba2c4af67 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.347231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46addd-e9ed-4533-bc7a-6d3ba2c4af67 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.347506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46addd-e9ed-4533-bc7a-6d3ba2c4af67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.366589 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9a46addd-e9ed-4533-bc7a-6d3ba2c4af67 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:41.368426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9a46addd-e9ed-4533-bc7a-6d3ba2c4af67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.369225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7f8fc-1891-4bab-b525-4ad649397367 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.420920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:41.432623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.433317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.433858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.434312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.435719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.437897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.450574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 881/1766] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:41.456299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 886/1767] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:41.460258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46addd-e9ed-4533-bc7a-6d3ba2c4af67 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.462457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.462627 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9a46addd-e9ed-4533-bc7a-6d3ba2c4af67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.464594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.478113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46addd-e9ed-4533-bc7a-6d3ba2c4af67 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.483917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b87d2-10dd-481a-89a6-ea6e729369b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.484579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b87d2-10dd-481a-89a6-ea6e729369b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.484857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b87d2-10dd-481a-89a6-ea6e729369b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.485131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b87d2-10dd-481a-89a6-ea6e729369b7 admin admin] PATH_INFO: `/v3/projects/130cc2faca61482fb6651e1ca3266b97/users/7071faea27344fac88a74f91af96b6f1/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.487233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b87d2-10dd-481a-89a6-ea6e729369b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=130cc2faca61482fb6651e1ca3266b97, user_id=7071faea27344fac88a74f91af96b6f1, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.490084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b87d2-10dd-481a-89a6-ea6e729369b7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.502640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 882/1768] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:41 2022] PUT /identity/v3/projects/130cc2faca61482fb6651e1ca3266b97/users/7071faea27344fac88a74f91af96b6f1/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:59:41.513695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eca2df-29d0-4140-9435-d1c1e0421d1c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.513924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eca2df-29d0-4140-9435-d1c1e0421d1c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.514364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eca2df-29d0-4140-9435-d1c1e0421d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.549875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-57eca2df-29d0-4140-9435-d1c1e0421d1c None None] MFA Rules not processed for user `7071faea27344fac88a74f91af96b6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:41.551134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-57eca2df-29d0-4140-9435-d1c1e0421d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.556910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0aa869-c1ba-4cc6-bb5d-4fd44c1043fe admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.616675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.617837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.618114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.618415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.650531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:41.667091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 883/1769] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:41.684216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eca2df-29d0-4140-9435-d1c1e0421d1c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.685761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-57eca2df-29d0-4140-9435-d1c1e0421d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.698221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.700046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.710488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 887/1770] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:59:41.754698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eca2df-29d0-4140-9435-d1c1e0421d1c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.758723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.759569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.759737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.760104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.761987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.827392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.828834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.839548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 884/1771] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:42.088089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:42.091396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:42.128953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a45da-b51a-47b8-82c8-01b1b4dcaffb service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:42.141155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:42.141794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:42.142071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:42.142390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] PATH_INFO: `/v3/projects/d3b16432751f4cd28120819f2c2b6701` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:42.143536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3b16432751f4cd28120819f2c2b6701)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:42.145427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:42.221740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:42.232340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3b16432751f4cd28120819f2c2b6701', 'request_id': 'req-2ecf6333-1912-4152-a909-9b6ed4e053ea'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.245307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3b16432751f4cd28120819f2c2b6701', 'request_id': 'req-2ecf6333-1912-4152-a909-9b6ed4e053ea'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.252126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3b16432751f4cd28120819f2c2b6701', 'request_id': 'req-2ecf6333-1912-4152-a909-9b6ed4e053ea'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.262581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] The token cache is being invalidate because project d3b16432751f4cd28120819f2c2b6701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:42.262870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.263557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.265051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 888/1772] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:42 2022] DELETE /identity/v3/projects/d3b16432751f4cd28120819f2c2b6701 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:42.268988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:42.280595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d3f3-5841-49e9-a1dc-b0eadbb31fb1 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:42.339217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:42.339863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:42.340146 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:42.340478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] PATH_INFO: `/v3/users/c34e75ba624c4915a039c80f9c9a22e3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:42.341651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34e75ba624c4915a039c80f9c9a22e3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:42.343683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:42.389094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34e75ba624c4915a039c80f9c9a22e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.394616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:42.396919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34e75ba624c4915a039c80f9c9a22e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.397749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34e75ba624c4915a039c80f9c9a22e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.398475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34e75ba624c4915a039c80f9c9a22e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.407193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-57524d75-e543-470c-859b-7abf066508d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34e75ba624c4915a039c80f9c9a22e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.425578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 885/1773] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:42 2022] DELETE /identity/v3/users/c34e75ba624c4915a039c80f9c9a22e3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:42.808361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:42.809921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:42.824245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf6333-1912-4152-a909-9b6ed4e053ea admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:42.881741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:42.882359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:42.882637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:42.882927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] PATH_INFO: `/v3/projects/a7d515641c6748939e8967adcceea34c` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:42.884579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7d515641c6748939e8967adcceea34c)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:42.886477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:42.931412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7d515641c6748939e8967adcceea34c', 'request_id': 'req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.938301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7d515641c6748939e8967adcceea34c', 'request_id': 'req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.944438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7d515641c6748939e8967adcceea34c', 'request_id': 'req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.955210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] The token cache is being invalidate because project a7d515641c6748939e8967adcceea34c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:42.955439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.956079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-90e5c240-bd1e-4903-afe6-0ffe413ac5dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:42.957557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 889/1774] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:42 2022] DELETE /identity/v3/projects/a7d515641c6748939e8967adcceea34c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:42.963045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f8f2b-85a8-40bf-ba6f-4bfa37f962e9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:42.963334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f8f2b-85a8-40bf-ba6f-4bfa37f962e9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:42.963609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f8f2b-85a8-40bf-ba6f-4bfa37f962e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:42.982690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-859f8f2b-85a8-40bf-ba6f-4bfa37f962e9 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:42.983929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-859f8f2b-85a8-40bf-ba6f-4bfa37f962e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:43.049316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 886/1775] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:43.057100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e05e8-6477-495b-ba2a-d2a02ffffdb3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.057391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e05e8-6477-495b-ba2a-d2a02ffffdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.057670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e05e8-6477-495b-ba2a-d2a02ffffdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.077671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ab3e05e8-6477-495b-ba2a-d2a02ffffdb3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:43.078883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ab3e05e8-6477-495b-ba2a-d2a02ffffdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:43.087554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 890/1776] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:43.091789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f8f2b-85a8-40bf-ba6f-4bfa37f962e9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.101486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f8f2b-85a8-40bf-ba6f-4bfa37f962e9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.106254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.106862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.107172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.107449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.108140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:59:43.108456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.110073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.126166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 887/1777] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:59:43.127420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e05e8-6477-495b-ba2a-d2a02ffffdb3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.137783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e05e8-6477-495b-ba2a-d2a02ffffdb3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.142799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f99-2f4b-4c41-a7f4-0c5973d25fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.143465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f99-2f4b-4c41-a7f4-0c5973d25fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.143742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f99-2f4b-4c41-a7f4-0c5973d25fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.144013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f99-2f4b-4c41-a7f4-0c5973d25fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.144647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6f99-2f4b-4c41-a7f4-0c5973d25fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.146561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6f99-2f4b-4c41-a7f4-0c5973d25fc2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.176074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 891/1778] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:43 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:43.180658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.190475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75451-bfb7-4f58-85d9-f951841faad5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.252167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01e3b1-47e4-4271-9ff6-f2835a56ed3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.252863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01e3b1-47e4-4271-9ff6-f2835a56ed3f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.253171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01e3b1-47e4-4271-9ff6-f2835a56ed3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.253443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01e3b1-47e4-4271-9ff6-f2835a56ed3f admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.254071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01e3b1-47e4-4271-9ff6-f2835a56ed3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.256008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01e3b1-47e4-4271-9ff6-f2835a56ed3f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.277777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 888/1779] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:43 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:43.281650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f99-2f4b-4c41-a7f4-0c5973d25fc2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.291877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f99-2f4b-4c41-a7f4-0c5973d25fc2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.296988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232683be-afa5-49b1-b30e-7f2b1de93e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.297588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232683be-afa5-49b1-b30e-7f2b1de93e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.297864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232683be-afa5-49b1-b30e-7f2b1de93e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.298133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232683be-afa5-49b1-b30e-7f2b1de93e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.298769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232683be-afa5-49b1-b30e-7f2b1de93e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.300368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232683be-afa5-49b1-b30e-7f2b1de93e6f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.314139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 892/1780] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:43.315438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01e3b1-47e4-4271-9ff6-f2835a56ed3f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.326092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01e3b1-47e4-4271-9ff6-f2835a56ed3f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.330882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8aadd4-75c0-45b2-b544-eaadee2886e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.331503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aadd4-75c0-45b2-b544-eaadee2886e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.331785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aadd4-75c0-45b2-b544-eaadee2886e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.332051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aadd4-75c0-45b2-b544-eaadee2886e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.332681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8aadd4-75c0-45b2-b544-eaadee2886e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.334241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8aadd4-75c0-45b2-b544-eaadee2886e3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.346256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 889/1781] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:43.350241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232683be-afa5-49b1-b30e-7f2b1de93e6f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.360215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232683be-afa5-49b1-b30e-7f2b1de93e6f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.364767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda972ce-e3d9-4e67-9ace-063370f4d97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.365400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda972ce-e3d9-4e67-9ace-063370f4d97d admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.365679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda972ce-e3d9-4e67-9ace-063370f4d97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.365943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda972ce-e3d9-4e67-9ace-063370f4d97d admin admin] PATH_INFO: `/v3/projects/6afba4a50e644c3590b1f6182ed44a55/users/6e0d0c48bad44e969f8efa1391c5d5ce/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.380743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda972ce-e3d9-4e67-9ace-063370f4d97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afba4a50e644c3590b1f6182ed44a55, user_id=6e0d0c48bad44e969f8efa1391c5d5ce, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.384428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda972ce-e3d9-4e67-9ace-063370f4d97d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.386866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:43.400404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 893/1782] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:43 2022] PUT /identity/v3/projects/6afba4a50e644c3590b1f6182ed44a55/users/6e0d0c48bad44e969f8efa1391c5d5ce/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:43.410462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26418f-93a7-4bd5-b2d1-6f2dad49ea05 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.410734 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26418f-93a7-4bd5-b2d1-6f2dad49ea05 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.411046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26418f-93a7-4bd5-b2d1-6f2dad49ea05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.440543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dc26418f-93a7-4bd5-b2d1-6f2dad49ea05 None None] MFA Rules not processed for user `6e0d0c48bad44e969f8efa1391c5d5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:43.441921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dc26418f-93a7-4bd5-b2d1-6f2dad49ea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:43.469142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:43.510653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 890/1783] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:43.515145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda972ce-e3d9-4e67-9ace-063370f4d97d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.577392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda972ce-e3d9-4e67-9ace-063370f4d97d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.581993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6396c-f6bd-464f-9a04-f394c1b74cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.582592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6396c-f6bd-464f-9a04-f394c1b74cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.582869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6396c-f6bd-464f-9a04-f394c1b74cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.583167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6396c-f6bd-464f-9a04-f394c1b74cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.583814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6396c-f6bd-464f-9a04-f394c1b74cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.585703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6396c-f6bd-464f-9a04-f394c1b74cfa admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.615535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 894/1784] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:43 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:59:43.617254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26418f-93a7-4bd5-b2d1-6f2dad49ea05 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.629345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26418f-93a7-4bd5-b2d1-6f2dad49ea05 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.693193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d2c5c-ddc0-4a92-9db5-d0ed870954a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.693821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d2c5c-ddc0-4a92-9db5-d0ed870954a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.694109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d2c5c-ddc0-4a92-9db5-d0ed870954a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.694485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d2c5c-ddc0-4a92-9db5-d0ed870954a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.695285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d2c5c-ddc0-4a92-9db5-d0ed870954a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.697253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d2c5c-ddc0-4a92-9db5-d0ed870954a3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.725950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 891/1785] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:43 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:43.727935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6396c-f6bd-464f-9a04-f394c1b74cfa admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.740130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6396c-f6bd-464f-9a04-f394c1b74cfa admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.746658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f450ef3-317a-48f1-936e-989031bcc6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.747307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450ef3-317a-48f1-936e-989031bcc6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.747590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450ef3-317a-48f1-936e-989031bcc6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.747916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450ef3-317a-48f1-936e-989031bcc6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.748574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f450ef3-317a-48f1-936e-989031bcc6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.750175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f450ef3-317a-48f1-936e-989031bcc6a8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.761454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 895/1786] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:43.765749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d2c5c-ddc0-4a92-9db5-d0ed870954a3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.776758 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d2c5c-ddc0-4a92-9db5-d0ed870954a3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.781584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43b45-5aa4-4351-aa22-36b2984b5f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.782244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43b45-5aa4-4351-aa22-36b2984b5f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.782528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43b45-5aa4-4351-aa22-36b2984b5f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.782804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43b45-5aa4-4351-aa22-36b2984b5f28 admin admin] PATH_INFO: `/v3/projects/3e07fa59f8f747bab55ff2d32b30d119/users/56b7528166564e788497216525efd242/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.801757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43b45-5aa4-4351-aa22-36b2984b5f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e07fa59f8f747bab55ff2d32b30d119, user_id=56b7528166564e788497216525efd242, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.804962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43b45-5aa4-4351-aa22-36b2984b5f28 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.837726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 892/1787] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:43 2022] PUT /identity/v3/projects/3e07fa59f8f747bab55ff2d32b30d119/users/56b7528166564e788497216525efd242/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:59:43.841636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f450ef3-317a-48f1-936e-989031bcc6a8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.856469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f450ef3-317a-48f1-936e-989031bcc6a8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.938415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3709b0-b4b4-4f0d-aa90-13aeccba0485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.939299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3709b0-b4b4-4f0d-aa90-13aeccba0485 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.939674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3709b0-b4b4-4f0d-aa90-13aeccba0485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.940008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3709b0-b4b4-4f0d-aa90-13aeccba0485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.940747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3709b0-b4b4-4f0d-aa90-13aeccba0485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.942561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3709b0-b4b4-4f0d-aa90-13aeccba0485 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.954458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 896/1788] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:59:43.958490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43b45-5aa4-4351-aa22-36b2984b5f28 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:43.968944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43b45-5aa4-4351-aa22-36b2984b5f28 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:43.973783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:43.974544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:43.975042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:43.975379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] PATH_INFO: `/v3/projects/3e07fa59f8f747bab55ff2d32b30d119/users/56b7528166564e788497216525efd242/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:43.977511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e07fa59f8f747bab55ff2d32b30d119, user_id=56b7528166564e788497216525efd242, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:43.980692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:43.992080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 893/1789] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:43 2022] PUT /identity/v3/projects/3e07fa59f8f747bab55ff2d32b30d119/users/56b7528166564e788497216525efd242/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:44.004532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8a06b-1b90-487d-8b7f-ce2b67e14522 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:44.004864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8a06b-1b90-487d-8b7f-ce2b67e14522 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:44.005149 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8a06b-1b90-487d-8b7f-ce2b67e14522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:44.036734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c7c8a06b-1b90-487d-8b7f-ce2b67e14522 None None] MFA Rules not processed for user `56b7528166564e788497216525efd242`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:44.037946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c7c8a06b-1b90-487d-8b7f-ce2b67e14522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:44.087444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:44.107508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 897/1790] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:44.119315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:44.121113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:44.180533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfec1a-4b1b-4b8c-9432-a6930f617ac3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:44.184337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccf1a4-2710-42ba-bbef-9b6a20e0bc9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:44.184940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccf1a4-2710-42ba-bbef-9b6a20e0bc9b service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:44.185217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccf1a4-2710-42ba-bbef-9b6a20e0bc9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:44.185483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccf1a4-2710-42ba-bbef-9b6a20e0bc9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:44.218569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:44.247039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccf1a4-2710-42ba-bbef-9b6a20e0bc9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:44.248869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccf1a4-2710-42ba-bbef-9b6a20e0bc9b service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:44.259249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 894/1791] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:47.629424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8a06b-1b90-487d-8b7f-ce2b67e14522 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:47.630843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c7c8a06b-1b90-487d-8b7f-ce2b67e14522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:47.784009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8a06b-1b90-487d-8b7f-ce2b67e14522 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:47.802570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:47.803685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:47.805832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:47.806272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] PATH_INFO: `/v3/users/9ee6c56817f4402ca489954ebfda89d3` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:47.809387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee6c56817f4402ca489954ebfda89d3)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:47.817286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:47.999396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee6c56817f4402ca489954ebfda89d3'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:48.045864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee6c56817f4402ca489954ebfda89d3'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:48.046940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee6c56817f4402ca489954ebfda89d3'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:48.047678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee6c56817f4402ca489954ebfda89d3'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:48.086613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee6c56817f4402ca489954ebfda89d3'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:48.116111 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 898/1792] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:47 2022] DELETE /identity/v3/users/9ee6c56817f4402ca489954ebfda89d3 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:48.120414 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9881c-951f-4dcd-a6be-7e1cf6885e6a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:48.121385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9881c-951f-4dcd-a6be-7e1cf6885e6a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:48.121910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9881c-951f-4dcd-a6be-7e1cf6885e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:48.151107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fbf9881c-951f-4dcd-a6be-7e1cf6885e6a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:48.152639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fbf9881c-951f-4dcd-a6be-7e1cf6885e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:48.262187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 895/1793] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:48.283918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:48.288844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:48.378998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd83a-03d1-4a0e-957f-0c08ce6abd7c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:48.383502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d230-187f-45ee-93c5-e6d27f2f03dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:48.384299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d230-187f-45ee-93c5-e6d27f2f03dd service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:48.384608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d230-187f-45ee-93c5-e6d27f2f03dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:48.384996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d230-187f-45ee-93c5-e6d27f2f03dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:48.409186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1d230-187f-45ee-93c5-e6d27f2f03dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:48.412866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1d230-187f-45ee-93c5-e6d27f2f03dd service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:48.430789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 899/1794] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:49.036836 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9881c-951f-4dcd-a6be-7e1cf6885e6a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:49.038577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fbf9881c-951f-4dcd-a6be-7e1cf6885e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:49.052769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9881c-951f-4dcd-a6be-7e1cf6885e6a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:49.085935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:49.087030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.087331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.088141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] PATH_INFO: `/v3/projects/a9de0fc88a2e45a4b2e5eb091f113848` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:49.097335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9de0fc88a2e45a4b2e5eb091f113848)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:49.100197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:49.207031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9de0fc88a2e45a4b2e5eb091f113848', 'request_id': 'req-5d319964-fecd-4260-834f-3bc3b3ec5569'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.217781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9de0fc88a2e45a4b2e5eb091f113848', 'request_id': 'req-5d319964-fecd-4260-834f-3bc3b3ec5569'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.230553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9de0fc88a2e45a4b2e5eb091f113848', 'request_id': 'req-5d319964-fecd-4260-834f-3bc3b3ec5569'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.245544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] The token cache is being invalidate because project a9de0fc88a2e45a4b2e5eb091f113848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:49.246116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.247096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5d319964-fecd-4260-834f-3bc3b3ec5569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.249455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 896/1795] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:49 2022] DELETE /identity/v3/projects/a9de0fc88a2e45a4b2e5eb091f113848 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:49.253436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d230-187f-45ee-93c5-e6d27f2f03dd service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:49.263438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d230-187f-45ee-93c5-e6d27f2f03dd service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:49.273917 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:49.340553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:49.341537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.342057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.342532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] PATH_INFO: `/v3/users/7071faea27344fac88a74f91af96b6f1` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:49.344065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7071faea27344fac88a74f91af96b6f1)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:49.346427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:49.411516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd9d92-3012-4afc-ad52-c0ab78cbcc97 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.418721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd9d92-3012-4afc-ad52-c0ab78cbcc97 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.419206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd9d92-3012-4afc-ad52-c0ab78cbcc97 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:49.421137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 897/1796] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 11:59:49 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:59:49.424205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7071faea27344fac88a74f91af96b6f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.433526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620b48b-365c-4fc3-b1fe-24f091394861 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.433878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620b48b-365c-4fc3-b1fe-24f091394861 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.434266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620b48b-365c-4fc3-b1fe-24f091394861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:49.441249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7071faea27344fac88a74f91af96b6f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.443611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7071faea27344fac88a74f91af96b6f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.445416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7071faea27344fac88a74f91af96b6f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.458694 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a620b48b-365c-4fc3-b1fe-24f091394861 None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:49.460846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a620b48b-365c-4fc3-b1fe-24f091394861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:49.474769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2d199dc7-0a88-4058-8a53-938a6b3a412a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7071faea27344fac88a74f91af96b6f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:49.507127 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 900/1797] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:49 2022] DELETE /identity/v3/users/7071faea27344fac88a74f91af96b6f1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:49.508403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab82895-6ed5-40d9-a9db-2581425b5429 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.508687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab82895-6ed5-40d9-a9db-2581425b5429 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.508968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab82895-6ed5-40d9-a9db-2581425b5429 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:49.510724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 901/1798] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 11:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:49.541696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:49.572830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 898/1799] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 11:59:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:49.579003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07f57-f770-4cb4-8047-ad639de342f1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.579003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07f57-f770-4cb4-8047-ad639de342f1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.579195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07f57-f770-4cb4-8047-ad639de342f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:49.593457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620b48b-365c-4fc3-b1fe-24f091394861 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:49.595658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a620b48b-365c-4fc3-b1fe-24f091394861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:49.606648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-62a07f57-f770-4cb4-8047-ad639de342f1 None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:49.608599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-62a07f57-f770-4cb4-8047-ad639de342f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:49.625601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620b48b-365c-4fc3-b1fe-24f091394861 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:49.862536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62665011-38c1-4fac-8208-d20451c4419e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:49.863837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62665011-38c1-4fac-8208-d20451c4419e service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.864117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62665011-38c1-4fac-8208-d20451c4419e service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.864387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62665011-38c1-4fac-8208-d20451c4419e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:49.873661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 902/1800] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 11:59:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:49.875262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62665011-38c1-4fac-8208-d20451c4419e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:49.876920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62665011-38c1-4fac-8208-d20451c4419e service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:49.892259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 899/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:59:50.277485 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a07f57-f770-4cb4-8047-ad639de342f1 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:50.278758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-62a07f57-f770-4cb4-8047-ad639de342f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:50.344001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:50.438793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a07f57-f770-4cb4-8047-ad639de342f1 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:50.453187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:50.453540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:50.453742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:50.453996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] PATH_INFO: `/v3/projects/130cc2faca61482fb6651e1ca3266b97` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:50.456836 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=130cc2faca61482fb6651e1ca3266b97)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:50.458808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:50.519583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130cc2faca61482fb6651e1ca3266b97', 'request_id': 'req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:50.529863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130cc2faca61482fb6651e1ca3266b97', 'request_id': 'req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:50.538045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130cc2faca61482fb6651e1ca3266b97', 'request_id': 'req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:50.554989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] The token cache is being invalidate because project 130cc2faca61482fb6651e1ca3266b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 11:59:50.555296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:50.556834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-61a7c92a-04a1-4f81-91d4-4055ce3b84bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:50.558589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 903/1802] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 11:59:50 2022] DELETE /identity/v3/projects/130cc2faca61482fb6651e1ca3266b97 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:50.563661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192bb81-023e-4851-a3c1-426f6bed123b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:50.564135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192bb81-023e-4851-a3c1-426f6bed123b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:50.564279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192bb81-023e-4851-a3c1-426f6bed123b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:50.589047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2192bb81-023e-4851-a3c1-426f6bed123b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:50.590980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2192bb81-023e-4851-a3c1-426f6bed123b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:50.699443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 900/1803] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:50.707277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beda98e-eca4-47b7-890d-957e2ae8207a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:50.707563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beda98e-eca4-47b7-890d-957e2ae8207a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:50.707831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beda98e-eca4-47b7-890d-957e2ae8207a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:50.741503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6beda98e-eca4-47b7-890d-957e2ae8207a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:50.744551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6beda98e-eca4-47b7-890d-957e2ae8207a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:50.756800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 904/1804] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:50.762966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192bb81-023e-4851-a3c1-426f6bed123b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:50.778360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192bb81-023e-4851-a3c1-426f6bed123b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:50.784547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:50.785249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:50.785547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:50.785828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:50.786518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:59:50.787009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:50.789037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:50.808320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 901/1805] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 11:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:59:50.814692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beda98e-eca4-47b7-890d-957e2ae8207a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:50.829471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beda98e-eca4-47b7-890d-957e2ae8207a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:50.835869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:50.836503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:50.836782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:50.837050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:50.837728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:50.839699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:50.886192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 905/1806] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 11:59:50 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:50.887930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:50.900885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197e9d9-5474-41c3-a7e1-6d652aa83617 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:50.968017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:50.975593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:51.062869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1be18e-4b38-4578-a63a-62d69339c881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:51.063572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1be18e-4b38-4578-a63a-62d69339c881 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:51.064840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1be18e-4b38-4578-a63a-62d69339c881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:51.065180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1be18e-4b38-4578-a63a-62d69339c881 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:51.066211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1be18e-4b38-4578-a63a-62d69339c881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:51.069782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1be18e-4b38-4578-a63a-62d69339c881 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:51.086476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:51.115337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa4208-2b17-4316-8f6c-8f6d7032b137 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:51.131282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:51.131282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:51.131282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:51.131282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:51.132041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:51.147783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 902/1807] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 11:59:50 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:59:51.149453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1be18e-4b38-4578-a63a-62d69339c881 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:51.152409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:51.155152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:51.164952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1be18e-4b38-4578-a63a-62d69339c881 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:51.177367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 906/1808] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:51.179205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae5108-63b1-4827-90e6-aa9a6261f694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:51.181520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5108-63b1-4827-90e6-aa9a6261f694 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:51.181851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5108-63b1-4827-90e6-aa9a6261f694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:51.182122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5108-63b1-4827-90e6-aa9a6261f694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:51.183092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae5108-63b1-4827-90e6-aa9a6261f694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:51.186348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae5108-63b1-4827-90e6-aa9a6261f694 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:51.209578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 903/1809] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:59:51.215029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:51.229333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a97ce7-2801-4368-b921-6fb7bc686bd8 service nova] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:51.234256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc34352-5e3c-4756-bb4a-bdfb5fdc44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:51.234982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34352-5e3c-4756-bb4a-bdfb5fdc44a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:51.235421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34352-5e3c-4756-bb4a-bdfb5fdc44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:51.235755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34352-5e3c-4756-bb4a-bdfb5fdc44a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:51.236484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc34352-5e3c-4756-bb4a-bdfb5fdc44a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:51.238279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc34352-5e3c-4756-bb4a-bdfb5fdc44a9 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:51.241510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:51.258123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae5108-63b1-4827-90e6-aa9a6261f694 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:51.269593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 907/1810] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 11:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:59:51.273121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae5108-63b1-4827-90e6-aa9a6261f694 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:51.278081 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:51.278747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:51.279181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:51.279457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] PATH_INFO: `/v3/projects/0c0e1719bd29452795963e73e0ed888f/users/32fe3636c0b4432fbb453892e68ac280/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:51.306036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0e1719bd29452795963e73e0ed888f, user_id=32fe3636c0b4432fbb453892e68ac280, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:51.311455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:51.330005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 904/1811] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 11:59:51 2022] PUT /identity/v3/projects/0c0e1719bd29452795963e73e0ed888f/users/32fe3636c0b4432fbb453892e68ac280/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:59:51.340777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506589ad-6c8a-4954-8e6b-4b4cb8c334cc None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:51.341328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506589ad-6c8a-4954-8e6b-4b4cb8c334cc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:51.342123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506589ad-6c8a-4954-8e6b-4b4cb8c334cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:51.379532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-506589ad-6c8a-4954-8e6b-4b4cb8c334cc None None] MFA Rules not processed for user `32fe3636c0b4432fbb453892e68ac280`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:51.381580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-506589ad-6c8a-4954-8e6b-4b4cb8c334cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:51.521074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 908/1812] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:51.533902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:51.536242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:51.581127 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506589ad-6c8a-4954-8e6b-4b4cb8c334cc None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:51.582546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-506589ad-6c8a-4954-8e6b-4b4cb8c334cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:51.670923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cabea-92e8-4cc9-a256-3ebf2041473f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:51.674870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6774fdd8b2e46b6a7d00919568f9fd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:51.675532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:51.676670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:51.676670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:51.682448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506589ad-6c8a-4954-8e6b-4b4cb8c334cc None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:51.686331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:51.687566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817588e-dbb2-4539-9e24-c33caa502d82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:51.688427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:51.688964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817588e-dbb2-4539-9e24-c33caa502d82 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:51.689427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817588e-dbb2-4539-9e24-c33caa502d82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:51.690011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817588e-dbb2-4539-9e24-c33caa502d82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:51.692780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e817588e-dbb2-4539-9e24-c33caa502d82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:51.734164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 905/1813] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:51.809407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817588e-dbb2-4539-9e24-c33caa502d82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:51.813021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817588e-dbb2-4539-9e24-c33caa502d82 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:51.851798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 909/1814] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:59.365399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:59.367105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:59.444102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad405a4-bd56-458f-861e-d6a63b8cf3b4 service glance] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:59.450412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:59.451215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:59.451601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:59.452006 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] PATH_INFO: `/v3/users/6e0d0c48bad44e969f8efa1391c5d5ce` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:59.453666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0d0c48bad44e969f8efa1391c5d5ce)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:59.456923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:59.513922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:59.521302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0d0c48bad44e969f8efa1391c5d5ce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:59.533459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0d0c48bad44e969f8efa1391c5d5ce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:59.534666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0d0c48bad44e969f8efa1391c5d5ce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:59.535424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0d0c48bad44e969f8efa1391c5d5ce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:59.544742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0d0c48bad44e969f8efa1391c5d5ce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 11:59:59.574612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 906/1815] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 11:59:59 2022] DELETE /identity/v3/users/6e0d0c48bad44e969f8efa1391c5d5ce => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:59:59.575949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ebbba-5adf-4abe-b617-32ae3bc84c60 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:59.576446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ebbba-5adf-4abe-b617-32ae3bc84c60 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:59.576543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ebbba-5adf-4abe-b617-32ae3bc84c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:59.605097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b85ebbba-5adf-4abe-b617-32ae3bc84c60 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:59.606299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b85ebbba-5adf-4abe-b617-32ae3bc84c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:59.647410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:59.681397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 910/1816] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 11:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:59.737573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:59.739154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:59.904797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9818ec-2649-4eca-ba06-51805b8cadd1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:59.914017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5ef4a-5864-46ce-a744-1696ae376b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:59.914928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ef4a-5864-46ce-a744-1696ae376b11 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:59.915526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ef4a-5864-46ce-a744-1696ae376b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:59.915947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ef4a-5864-46ce-a744-1696ae376b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:59.975353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5ef4a-5864-46ce-a744-1696ae376b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:59.977082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5ef4a-5864-46ce-a744-1696ae376b11 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:59.996107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 907/1817] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:00.443648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ebbba-5adf-4abe-b617-32ae3bc84c60 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:00.445653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b85ebbba-5adf-4abe-b617-32ae3bc84c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:00.483985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ebbba-5adf-4abe-b617-32ae3bc84c60 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:00.489429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:00.490388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:00.490736 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:00.491102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] PATH_INFO: `/v3/projects/6afba4a50e644c3590b1f6182ed44a55` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:00.492419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6afba4a50e644c3590b1f6182ed44a55)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:00.495043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:00.549070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6afba4a50e644c3590b1f6182ed44a55', 'request_id': 'req-41feaecd-9148-4ac3-8ebb-40f59c2bf173'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.561633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6afba4a50e644c3590b1f6182ed44a55', 'request_id': 'req-41feaecd-9148-4ac3-8ebb-40f59c2bf173'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.568644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6afba4a50e644c3590b1f6182ed44a55', 'request_id': 'req-41feaecd-9148-4ac3-8ebb-40f59c2bf173'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.580384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] The token cache is being invalidate because project 6afba4a50e644c3590b1f6182ed44a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:00.581031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.581432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.583007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 911/1818] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:00 2022] DELETE /identity/v3/projects/6afba4a50e644c3590b1f6182ed44a55 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:00.586858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5ef4a-5864-46ce-a744-1696ae376b11 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:00.599278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5ef4a-5864-46ce-a744-1696ae376b11 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:00.685162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:00.687878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:00.688436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:00.688920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] PATH_INFO: `/v3/users/56b7528166564e788497216525efd242` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:00.691139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b7528166564e788497216525efd242)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:00.694049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:00.774435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b7528166564e788497216525efd242'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.785382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b7528166564e788497216525efd242'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.786489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b7528166564e788497216525efd242'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.787251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b7528166564e788497216525efd242'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.796686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b7528166564e788497216525efd242'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:00.798541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:00.836970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 908/1819] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:00 2022] DELETE /identity/v3/users/56b7528166564e788497216525efd242 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:01.821026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:01.824662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:01.841097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41feaecd-9148-4ac3-8ebb-40f59c2bf173 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:01.921073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:01.921845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:01.922201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:01.922584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] PATH_INFO: `/v3/users/055e7bc4c72d4bf3b2f172d599757991` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:01.925474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=055e7bc4c72d4bf3b2f172d599757991)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:01.927733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:01.938349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:01.939928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:01.964206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535267fc-2afa-489f-9aa8-bdc27c4a78e8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:01.970360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:01.972077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:01.972426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:01.972767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] PATH_INFO: `/v3/projects/3e07fa59f8f747bab55ff2d32b30d119` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:01.973966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e07fa59f8f747bab55ff2d32b30d119)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:01.975974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:02.008028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055e7bc4c72d4bf3b2f172d599757991'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.016398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055e7bc4c72d4bf3b2f172d599757991'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.017094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055e7bc4c72d4bf3b2f172d599757991'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.017779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055e7bc4c72d4bf3b2f172d599757991'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.027492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-33c907d8-ce73-4fb8-b7cb-e2a5a8a67b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055e7bc4c72d4bf3b2f172d599757991'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.029551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:02.046924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e07fa59f8f747bab55ff2d32b30d119', 'request_id': 'req-065a60b6-678b-418c-81d7-8eb84f683063'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.054623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 912/1820] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:01 2022] DELETE /identity/v3/users/055e7bc4c72d4bf3b2f172d599757991 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:02.058952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e07fa59f8f747bab55ff2d32b30d119', 'request_id': 'req-065a60b6-678b-418c-81d7-8eb84f683063'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.059465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc5af-dcd7-4519-92b7-a72bf5a51fdb None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.059977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc5af-dcd7-4519-92b7-a72bf5a51fdb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.060631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc5af-dcd7-4519-92b7-a72bf5a51fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.066689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e07fa59f8f747bab55ff2d32b30d119', 'request_id': 'req-065a60b6-678b-418c-81d7-8eb84f683063'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.081802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] The token cache is being invalidate because project 3e07fa59f8f747bab55ff2d32b30d119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:02.082247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.083083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-065a60b6-678b-418c-81d7-8eb84f683063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:02.083709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-508bc5af-dcd7-4519-92b7-a72bf5a51fdb None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:02.084676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 909/1821] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:01 2022] DELETE /identity/v3/projects/3e07fa59f8f747bab55ff2d32b30d119 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:02.084970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-508bc5af-dcd7-4519-92b7-a72bf5a51fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:02.091118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb37f3-0efb-4590-a27f-a72f8409f778 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.091413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb37f3-0efb-4590-a27f-a72f8409f778 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.091684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb37f3-0efb-4590-a27f-a72f8409f778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.118924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dedb37f3-0efb-4590-a27f-a72f8409f778 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:02.118924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dedb37f3-0efb-4590-a27f-a72f8409f778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:02.161677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 913/1822] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:02.170935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bc5af-dcd7-4519-92b7-a72bf5a51fdb None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.174362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-508bc5af-dcd7-4519-92b7-a72bf5a51fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:02.195303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 910/1823] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:02.204172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0e80d-32c2-432b-99d0-51dbde78c37a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.204547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0e80d-32c2-432b-99d0-51dbde78c37a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.205287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0e80d-32c2-432b-99d0-51dbde78c37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.231055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-61b0e80d-32c2-432b-99d0-51dbde78c37a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:02.232552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-61b0e80d-32c2-432b-99d0-51dbde78c37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:02.250657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 911/1824] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:02.252219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bc5af-dcd7-4519-92b7-a72bf5a51fdb None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:02.255009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0e80d-32c2-432b-99d0-51dbde78c37a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.257939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00ea7f-44f4-4199-aed7-69fbb6b9010f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:02.258567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ea7f-44f4-4199-aed7-69fbb6b9010f service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.258845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ea7f-44f4-4199-aed7-69fbb6b9010f service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.259173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ea7f-44f4-4199-aed7-69fbb6b9010f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.265676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0e80d-32c2-432b-99d0-51dbde78c37a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:02.269218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00ea7f-44f4-4199-aed7-69fbb6b9010f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:02.270778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00ea7f-44f4-4199-aed7-69fbb6b9010f service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:02.272021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:02.273998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.274377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.274934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.276860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:02.276860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:02.277746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:02.282967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 914/1825] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:02.293120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 912/1826] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:02.297306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00ea7f-44f4-4199-aed7-69fbb6b9010f service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.307964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00ea7f-44f4-4199-aed7-69fbb6b9010f service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:02.312911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b87cfd-0b0e-4d1e-ae83-1e001ab0ba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:02.313670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b87cfd-0b0e-4d1e-ae83-1e001ab0ba6a admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.313913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b87cfd-0b0e-4d1e-ae83-1e001ab0ba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.314217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b87cfd-0b0e-4d1e-ae83-1e001ab0ba6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.314873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b87cfd-0b0e-4d1e-ae83-1e001ab0ba6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:02.316758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b87cfd-0b0e-4d1e-ae83-1e001ab0ba6a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:02.359978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 915/1827] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:02 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:02.360695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.381963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7c9c7-595a-4494-8300-198133a8d10b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:02.477798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:02.531814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9957e-696b-4610-9228-02e27fa8090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:02.531814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9957e-696b-4610-9228-02e27fa8090f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.532021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9957e-696b-4610-9228-02e27fa8090f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.532350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9957e-696b-4610-9228-02e27fa8090f admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.532936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9957e-696b-4610-9228-02e27fa8090f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:02.534836 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9957e-696b-4610-9228-02e27fa8090f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:02.586678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 913/1828] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:02 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:02.591440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b87cfd-0b0e-4d1e-ae83-1e001ab0ba6a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.604690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b87cfd-0b0e-4d1e-ae83-1e001ab0ba6a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:02.609969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56346-1d6a-4576-a8f9-af1ed672d8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:02.610610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56346-1d6a-4576-a8f9-af1ed672d8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.613300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56346-1d6a-4576-a8f9-af1ed672d8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.613300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56346-1d6a-4576-a8f9-af1ed672d8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.613300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a56346-1d6a-4576-a8f9-af1ed672d8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:02.613611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a56346-1d6a-4576-a8f9-af1ed672d8d3 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:02.625101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 916/1829] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:02.628799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9957e-696b-4610-9228-02e27fa8090f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.641523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9957e-696b-4610-9228-02e27fa8090f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:02.648078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31522c31-75b6-4695-91c6-7c313a0e0b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:02.648078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31522c31-75b6-4695-91c6-7c313a0e0b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.648078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31522c31-75b6-4695-91c6-7c313a0e0b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.648078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31522c31-75b6-4695-91c6-7c313a0e0b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.649017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31522c31-75b6-4695-91c6-7c313a0e0b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:02.651070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31522c31-75b6-4695-91c6-7c313a0e0b59 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:02.662735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 914/1830] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:02.667910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56346-1d6a-4576-a8f9-af1ed672d8d3 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.678364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56346-1d6a-4576-a8f9-af1ed672d8d3 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:02.685060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:02.685868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.686254 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.686632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] PATH_INFO: `/v3/projects/9ae673543def4e7ba74edd07fff072ca/users/01dfb64c3ded4e8bac973c1b32c30774/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.703490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae673543def4e7ba74edd07fff072ca, user_id=01dfb64c3ded4e8bac973c1b32c30774, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:02.715904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:02.736850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 917/1831] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:02 2022] PUT /identity/v3/projects/9ae673543def4e7ba74edd07fff072ca/users/01dfb64c3ded4e8bac973c1b32c30774/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:02.749445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc164886-3182-4abc-87b3-6270248e2593 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.749804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc164886-3182-4abc-87b3-6270248e2593 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.750100 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc164886-3182-4abc-87b3-6270248e2593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.783161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bc164886-3182-4abc-87b3-6270248e2593 None None] MFA Rules not processed for user `01dfb64c3ded4e8bac973c1b32c30774`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:02.784407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bc164886-3182-4abc-87b3-6270248e2593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:02.900575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.903194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:02.927960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 915/1832] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:02.941812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc164886-3182-4abc-87b3-6270248e2593 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:02.944448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bc164886-3182-4abc-87b3-6270248e2593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:02.965984 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:03.038954 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc164886-3182-4abc-87b3-6270248e2593 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:03.042692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dba0e4-fd96-4b31-90e1-ac83aa7335ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:03.043370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dba0e4-fd96-4b31-90e1-ac83aa7335ed service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.043647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dba0e4-fd96-4b31-90e1-ac83aa7335ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.043921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dba0e4-fd96-4b31-90e1-ac83aa7335ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.046937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a433ed0-3790-4ba2-9a3b-fbce5edec8a6 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:03.052144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:03.053349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dba0e4-fd96-4b31-90e1-ac83aa7335ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:03.053498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.053646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.054186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] PATH_INFO: `/v3/projects/8c0d8f662c9c493db9529a6912881684` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.055396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0d8f662c9c493db9529a6912881684)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:03.055512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dba0e4-fd96-4b31-90e1-ac83aa7335ed service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:03.057432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:03.075805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 916/1833] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:03.090872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dba0e4-fd96-4b31-90e1-ac83aa7335ed service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:03.108600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dba0e4-fd96-4b31-90e1-ac83aa7335ed service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:03.117782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0d8f662c9c493db9529a6912881684', 'request_id': 'req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.127619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0d8f662c9c493db9529a6912881684', 'request_id': 'req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.136042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0d8f662c9c493db9529a6912881684', 'request_id': 'req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.149180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] The token cache is being invalidate because project 8c0d8f662c9c493db9529a6912881684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:03.149519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.150825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.155594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 918/1834] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:02 2022] DELETE /identity/v3/projects/8c0d8f662c9c493db9529a6912881684 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:03.156670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:03.158366 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:03.177494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:03.194792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:03.195828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.199261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.199685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] PATH_INFO: `/v3/users/01dfb64c3ded4e8bac973c1b32c30774` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.200952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01dfb64c3ded4e8bac973c1b32c30774)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:03.202963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:03.244452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b9d0-9325-49ca-84e5-2c638ba5a0d6 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:03.249294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:03.249926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.250206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.250477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] PATH_INFO: `/v3/users/ee56bc99373a4e29ad0376135861f008` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.251683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee56bc99373a4e29ad0376135861f008)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:03.253683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:03.257732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01dfb64c3ded4e8bac973c1b32c30774'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.266277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01dfb64c3ded4e8bac973c1b32c30774'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.267182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01dfb64c3ded4e8bac973c1b32c30774'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.267895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01dfb64c3ded4e8bac973c1b32c30774'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.275649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e00b17c3-1d05-47b5-a0a0-94aa61e866d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01dfb64c3ded4e8bac973c1b32c30774'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.307608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee56bc99373a4e29ad0376135861f008'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.308129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 917/1835] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:03 2022] DELETE /identity/v3/users/01dfb64c3ded4e8bac973c1b32c30774 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:03.312511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90879c39-1562-4ab8-bf6f-b286b28a61db None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.312798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90879c39-1562-4ab8-bf6f-b286b28a61db None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.313071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90879c39-1562-4ab8-bf6f-b286b28a61db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.316337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee56bc99373a4e29ad0376135861f008'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.317069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee56bc99373a4e29ad0376135861f008'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.317696 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee56bc99373a4e29ad0376135861f008'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.320368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:03.328679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee56bc99373a4e29ad0376135861f008'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.336903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-90879c39-1562-4ab8-bf6f-b286b28a61db None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:03.338669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-90879c39-1562-4ab8-bf6f-b286b28a61db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:03.352241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 919/1836] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:03 2022] DELETE /identity/v3/users/ee56bc99373a4e29ad0376135861f008 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:03.423753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 918/1837] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:03.437534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:03.439468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:03.508991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b0bbc-c295-4f73-b3a4-aa4dbaf93429 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:03.513633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52cc06-fd02-4eae-8458-112c3b783681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:03.514458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52cc06-fd02-4eae-8458-112c3b783681 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.514809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52cc06-fd02-4eae-8458-112c3b783681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.515181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52cc06-fd02-4eae-8458-112c3b783681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.611735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52cc06-fd02-4eae-8458-112c3b783681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:03.620038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52cc06-fd02-4eae-8458-112c3b783681 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:03.636079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 920/1838] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:03.685311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90879c39-1562-4ab8-bf6f-b286b28a61db None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:03.688878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-90879c39-1562-4ab8-bf6f-b286b28a61db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:03.721270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90879c39-1562-4ab8-bf6f-b286b28a61db None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:03.729846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:03.730653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.731082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.731448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] PATH_INFO: `/v3/projects/1d7ccbbbc8e54ca58f168e342062bb56` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.733156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7ccbbbc8e54ca58f168e342062bb56)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:03.735499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:03.788120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7ccbbbc8e54ca58f168e342062bb56', 'request_id': 'req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.796068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7ccbbbc8e54ca58f168e342062bb56', 'request_id': 'req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.802864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7ccbbbc8e54ca58f168e342062bb56', 'request_id': 'req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.810742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:03.821317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] The token cache is being invalidate because project 1d7ccbbbc8e54ca58f168e342062bb56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:03.822801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.824388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e9a04ff6-cfdd-4a55-9e31-d6b83261f134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:03.835195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 919/1839] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:03 2022] DELETE /identity/v3/projects/1d7ccbbbc8e54ca58f168e342062bb56 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:03.835311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f8c3-5aab-4441-bc9e-f58f5ce2c46d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.835311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f8c3-5aab-4441-bc9e-f58f5ce2c46d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.835311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f8c3-5aab-4441-bc9e-f58f5ce2c46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.857664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b910f8c3-5aab-4441-bc9e-f58f5ce2c46d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:03.859064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b910f8c3-5aab-4441-bc9e-f58f5ce2c46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:03.923199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 921/1840] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:03.931306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb6c7c-45d7-4851-90f2-4bef79708687 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:03.931672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb6c7c-45d7-4851-90f2-4bef79708687 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:03.932010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb6c7c-45d7-4851-90f2-4bef79708687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:03.954242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-79cb6c7c-45d7-4851-90f2-4bef79708687 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:03.955391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-79cb6c7c-45d7-4851-90f2-4bef79708687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:03.969292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 920/1841] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:03.970488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910f8c3-5aab-4441-bc9e-f58f5ce2c46d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:03.994926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910f8c3-5aab-4441-bc9e-f58f5ce2c46d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.001599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.002303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.002823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.003135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.003868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:04.004237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.006092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.021150 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 922/1842] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:00:04.025217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb6c7c-45d7-4851-90f2-4bef79708687 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.034586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb6c7c-45d7-4851-90f2-4bef79708687 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.040608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c3158-4aae-43ed-be20-4697b71730c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.041390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3158-4aae-43ed-be20-4697b71730c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.041810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3158-4aae-43ed-be20-4697b71730c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.042242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3158-4aae-43ed-be20-4697b71730c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.043059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c3158-4aae-43ed-be20-4697b71730c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.045241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c3158-4aae-43ed-be20-4697b71730c6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.057639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.060518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:04.085186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7cba9-6639-490b-a43c-6b739ecb0c5f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.093015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.093771 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.094757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.094987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] PATH_INFO: `/v3/projects/9ae673543def4e7ba74edd07fff072ca` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.095094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 921/1843] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:04.096452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae673543def4e7ba74edd07fff072ca)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.098602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.099154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c3158-4aae-43ed-be20-4697b71730c6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.109431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:04.160046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae673543def4e7ba74edd07fff072ca', 'request_id': 'req-1124dd64-49ac-43e1-b88a-8f42e2fa465d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:04.179843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae673543def4e7ba74edd07fff072ca', 'request_id': 'req-1124dd64-49ac-43e1-b88a-8f42e2fa465d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:04.194046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae673543def4e7ba74edd07fff072ca', 'request_id': 'req-1124dd64-49ac-43e1-b88a-8f42e2fa465d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:04.195666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c3158-4aae-43ed-be20-4697b71730c6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.225341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] The token cache is being invalidate because project 9ae673543def4e7ba74edd07fff072ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:04.225730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:04.231861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1124dd64-49ac-43e1-b88a-8f42e2fa465d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:04.234297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 923/1844] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:04 2022] DELETE /identity/v3/projects/9ae673543def4e7ba74edd07fff072ca => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:04.239019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac129a9-7d69-4fae-b1b7-10c021698a3f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.239760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac129a9-7d69-4fae-b1b7-10c021698a3f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.240201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac129a9-7d69-4fae-b1b7-10c021698a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.266637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-4ac129a9-7d69-4fae-b1b7-10c021698a3f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:04.268118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4ac129a9-7d69-4fae-b1b7-10c021698a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:04.303546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.304496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.304786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.305057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.306088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.308029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.329448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 922/1845] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:04.333509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.335090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:04.335201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 924/1846] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:04.346024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7bbd0-6f45-46d2-9788-7b314209dc30 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.346316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7bbd0-6f45-46d2-9788-7b314209dc30 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.346800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7bbd0-6f45-46d2-9788-7b314209dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.349555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1b0dc-6bd0-4cd6-9e01-58d0bd3b3ba8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.355089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e82adc-efea-4284-a4aa-2e46076f7e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.355739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e82adc-efea-4284-a4aa-2e46076f7e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.356023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e82adc-efea-4284-a4aa-2e46076f7e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.356298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e82adc-efea-4284-a4aa-2e46076f7e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.356977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e82adc-efea-4284-a4aa-2e46076f7e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.358606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e82adc-efea-4284-a4aa-2e46076f7e34 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.367699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-17c7bbd0-6f45-46d2-9788-7b314209dc30 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:04.368928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-17c7bbd0-6f45-46d2-9788-7b314209dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:04.372342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 923/1847] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:04.373293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e82adc-efea-4284-a4aa-2e46076f7e34 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.376796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 925/1848] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:04.380762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7bbd0-6f45-46d2-9788-7b314209dc30 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.383524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e82adc-efea-4284-a4aa-2e46076f7e34 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.389150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5b30d-902a-43e1-82eb-53867d6747b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.389741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5b30d-902a-43e1-82eb-53867d6747b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.390023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5b30d-902a-43e1-82eb-53867d6747b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.390137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7bbd0-6f45-46d2-9788-7b314209dc30 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.390301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5b30d-902a-43e1-82eb-53867d6747b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.390959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5b30d-902a-43e1-82eb-53867d6747b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.392528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5b30d-902a-43e1-82eb-53867d6747b5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.394555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.395169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.395446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.395715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.396349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:04.396649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.398209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.406721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 924/1849] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:04.410817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5b30d-902a-43e1-82eb-53867d6747b5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.417789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 926/1850] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:04.418783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.421855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5b30d-902a-43e1-82eb-53867d6747b5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.426783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9b973-45b9-4b75-8015-cc7784f78342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.427562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b973-45b9-4b75-8015-cc7784f78342 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.427848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b973-45b9-4b75-8015-cc7784f78342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.428123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b973-45b9-4b75-8015-cc7784f78342 admin admin] PATH_INFO: `/v3/projects/0ea97f8f601d4d059e1cd84fed8e938c/users/47c7c127f3364c91a9f264c3b8da6248/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.428672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f47236-d455-4942-8684-f2b79d52be21 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.435184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a128a9-a2f3-4afc-a312-fbc7bb54e592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.435794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a128a9-a2f3-4afc-a312-fbc7bb54e592 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.436074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a128a9-a2f3-4afc-a312-fbc7bb54e592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.436346 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a128a9-a2f3-4afc-a312-fbc7bb54e592 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.437571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a128a9-a2f3-4afc-a312-fbc7bb54e592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.439613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a128a9-a2f3-4afc-a312-fbc7bb54e592 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.443967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9b973-45b9-4b75-8015-cc7784f78342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea97f8f601d4d059e1cd84fed8e938c, user_id=47c7c127f3364c91a9f264c3b8da6248, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.447798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9b973-45b9-4b75-8015-cc7784f78342 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.461211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 925/1851] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:04 2022] PUT /identity/v3/projects/0ea97f8f601d4d059e1cd84fed8e938c/users/47c7c127f3364c91a9f264c3b8da6248/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:04.470917 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 927/1852] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:04.474344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9b973-45b9-4b75-8015-cc7784f78342 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.476488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef4382-c68e-4227-ae28-30baca2d8d66 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.476772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef4382-c68e-4227-ae28-30baca2d8d66 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.477043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef4382-c68e-4227-ae28-30baca2d8d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.497632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:04.506692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-16ef4382-c68e-4227-ae28-30baca2d8d66 None None] MFA Rules not processed for user `47c7c127f3364c91a9f264c3b8da6248`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:04.507926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-16ef4382-c68e-4227-ae28-30baca2d8d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:04.538779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9b973-45b9-4b75-8015-cc7784f78342 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.559178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:04.581767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 928/1853] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:04.587042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef4382-c68e-4227-ae28-30baca2d8d66 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.608107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b93c48-fb02-4996-a606-1c348270139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.608810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b93c48-fb02-4996-a606-1c348270139d admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.609092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b93c48-fb02-4996-a606-1c348270139d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.609366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b93c48-fb02-4996-a606-1c348270139d admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.609992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b93c48-fb02-4996-a606-1c348270139d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.611942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b93c48-fb02-4996-a606-1c348270139d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.637328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 926/1854] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:04.638485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b93c48-fb02-4996-a606-1c348270139d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.650464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b93c48-fb02-4996-a606-1c348270139d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.655182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e9d8c-a80d-497c-b509-82f65bf513d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.655805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9d8c-a80d-497c-b509-82f65bf513d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.656085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9d8c-a80d-497c-b509-82f65bf513d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.656520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9d8c-a80d-497c-b509-82f65bf513d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.657240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e9d8c-a80d-497c-b509-82f65bf513d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.658821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e9d8c-a80d-497c-b509-82f65bf513d0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.663805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef4382-c68e-4227-ae28-30baca2d8d66 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.668459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f6a5c-5b2a-44ae-a937-c758bac4db5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.669070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f6a5c-5b2a-44ae-a937-c758bac4db5e admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.669188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 927/1855] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:04.669384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f6a5c-5b2a-44ae-a937-c758bac4db5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.669652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f6a5c-5b2a-44ae-a937-c758bac4db5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.670278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f6a5c-5b2a-44ae-a937-c758bac4db5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.672216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f6a5c-5b2a-44ae-a937-c758bac4db5e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.672929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e9d8c-a80d-497c-b509-82f65bf513d0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.682837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e9d8c-a80d-497c-b509-82f65bf513d0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.687843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfb253-b3be-4a72-accc-e14e921c1eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.688428 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb253-b3be-4a72-accc-e14e921c1eed admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.688715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb253-b3be-4a72-accc-e14e921c1eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.689049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb253-b3be-4a72-accc-e14e921c1eed admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.689612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfb253-b3be-4a72-accc-e14e921c1eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.691218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfb253-b3be-4a72-accc-e14e921c1eed admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.700027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 929/1856] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:04.706289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 928/1857] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:04.706490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f6a5c-5b2a-44ae-a937-c758bac4db5e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.708406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfb253-b3be-4a72-accc-e14e921c1eed admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.721492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfb253-b3be-4a72-accc-e14e921c1eed admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.772224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f6a5c-5b2a-44ae-a937-c758bac4db5e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.777322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c4c5-5487-4b8b-b888-80ee5b8113fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.777953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c4c5-5487-4b8b-b888-80ee5b8113fe admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.778226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c4c5-5487-4b8b-b888-80ee5b8113fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.778488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c4c5-5487-4b8b-b888-80ee5b8113fe admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.779227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74c4c5-5487-4b8b-b888-80ee5b8113fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.781243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74c4c5-5487-4b8b-b888-80ee5b8113fe admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.781752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce5568-7ff0-45b0-8c1e-b5df8ca58e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.782418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce5568-7ff0-45b0-8c1e-b5df8ca58e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.782732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce5568-7ff0-45b0-8c1e-b5df8ca58e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.782998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce5568-7ff0-45b0-8c1e-b5df8ca58e29 admin admin] PATH_INFO: `/v3/projects/fb07c878a90944a5874993d0d5d57b78/users/8e63f18616d04c0a8fa5eb64582a80c4/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.797547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce5568-7ff0-45b0-8c1e-b5df8ca58e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb07c878a90944a5874993d0d5d57b78, user_id=8e63f18616d04c0a8fa5eb64582a80c4, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.800225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce5568-7ff0-45b0-8c1e-b5df8ca58e29 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.802500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 930/1858] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:04.807659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c4c5-5487-4b8b-b888-80ee5b8113fe admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.817141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 929/1859] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:04 2022] PUT /identity/v3/projects/fb07c878a90944a5874993d0d5d57b78/users/8e63f18616d04c0a8fa5eb64582a80c4/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:04.818613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c4c5-5487-4b8b-b888-80ee5b8113fe admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.832815 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b2799-3f86-489b-86db-d6a448015647 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.832815 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b2799-3f86-489b-86db-d6a448015647 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.832815 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b2799-3f86-489b-86db-d6a448015647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.862469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b89b2799-3f86-489b-86db-d6a448015647 None None] MFA Rules not processed for user `8e63f18616d04c0a8fa5eb64582a80c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:04.863691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b89b2799-3f86-489b-86db-d6a448015647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:04.869538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:04.900241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797e86c-335f-4965-a4d0-bf7fbae777bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.900878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797e86c-335f-4965-a4d0-bf7fbae777bb admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.901144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797e86c-335f-4965-a4d0-bf7fbae777bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.901419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797e86c-335f-4965-a4d0-bf7fbae777bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.902060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797e86c-335f-4965-a4d0-bf7fbae777bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.903923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797e86c-335f-4965-a4d0-bf7fbae777bb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:04.905454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:04.922558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 931/1860] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:04.923579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797e86c-335f-4965-a4d0-bf7fbae777bb admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.935828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797e86c-335f-4965-a4d0-bf7fbae777bb admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:04.941922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fefa8-1787-45c3-be90-0196a2c254db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:04.942714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fefa8-1787-45c3-be90-0196a2c254db admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:04.943096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fefa8-1787-45c3-be90-0196a2c254db admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:04.943367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fefa8-1787-45c3-be90-0196a2c254db admin admin] PATH_INFO: `/v3/projects/dc2c6f9df9bc4bf18ce7627b5605f1da/users/6c7fe421e2d2426591c8a9acfd3bc160/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:04.962426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 930/1861] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:04.971867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fefa8-1787-45c3-be90-0196a2c254db admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2c6f9df9bc4bf18ce7627b5605f1da, user_id=6c7fe421e2d2426591c8a9acfd3bc160, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:04.975166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b2799-3f86-489b-86db-d6a448015647 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:04.976654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b89b2799-3f86-489b-86db-d6a448015647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:04.977461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fefa8-1787-45c3-be90-0196a2c254db admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:05.000457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 932/1862] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:04 2022] PUT /identity/v3/projects/dc2c6f9df9bc4bf18ce7627b5605f1da/users/6c7fe421e2d2426591c8a9acfd3bc160/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:05.002224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fefa8-1787-45c3-be90-0196a2c254db admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:05.049221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b2799-3f86-489b-86db-d6a448015647 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:05.091511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fefa8-1787-45c3-be90-0196a2c254db admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:05.097096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a34966-adce-42c0-b57e-a066008a522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:05.097998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a34966-adce-42c0-b57e-a066008a522d admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:05.098282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a34966-adce-42c0-b57e-a066008a522d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:05.098553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a34966-adce-42c0-b57e-a066008a522d admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:05.099511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a34966-adce-42c0-b57e-a066008a522d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:05.101582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a34966-adce-42c0-b57e-a066008a522d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:05.120278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 933/1863] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:05.121574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a34966-adce-42c0-b57e-a066008a522d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:05.135902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a34966-adce-42c0-b57e-a066008a522d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:05.141343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7e655-a14a-4121-9dae-2d12a0e02b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:05.142029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7e655-a14a-4121-9dae-2d12a0e02b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:05.142307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7e655-a14a-4121-9dae-2d12a0e02b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:05.142577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7e655-a14a-4121-9dae-2d12a0e02b55 admin admin] PATH_INFO: `/v3/projects/dc2c6f9df9bc4bf18ce7627b5605f1da/users/6c7fe421e2d2426591c8a9acfd3bc160/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:05.144616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7e655-a14a-4121-9dae-2d12a0e02b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2c6f9df9bc4bf18ce7627b5605f1da, user_id=6c7fe421e2d2426591c8a9acfd3bc160, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:05.147435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7e655-a14a-4121-9dae-2d12a0e02b55 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:05.152468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6774fdd8b2e46b6a7d00919568f9fd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:05.153338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:05.153821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:05.154304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:05.184870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 934/1864] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:05 2022] PUT /identity/v3/projects/dc2c6f9df9bc4bf18ce7627b5605f1da/users/6c7fe421e2d2426591c8a9acfd3bc160/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:05.197145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174072c-ad7e-4b20-89d6-cb2965d26849 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:05.197437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174072c-ad7e-4b20-89d6-cb2965d26849 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:05.197714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174072c-ad7e-4b20-89d6-cb2965d26849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:05.232746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b174072c-ad7e-4b20-89d6-cb2965d26849 None None] MFA Rules not processed for user `6c7fe421e2d2426591c8a9acfd3bc160`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:05.233903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b174072c-ad7e-4b20-89d6-cb2965d26849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:05.252976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:05.254948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:05.267846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 931/1865] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:05.360377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 935/1866] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:05.532477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:05.540452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:05.565016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:05.624763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab6929-6dd1-420b-ab05-87285103dcee service glance] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:05.702304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2beb2-1566-43ad-8430-91f3aac5c837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:05.703248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2beb2-1566-43ad-8430-91f3aac5c837 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:05.703661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2beb2-1566-43ad-8430-91f3aac5c837 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:05.704200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2beb2-1566-43ad-8430-91f3aac5c837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:05.718531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2beb2-1566-43ad-8430-91f3aac5c837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:05.720393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2beb2-1566-43ad-8430-91f3aac5c837 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:05.733609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 932/1867] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:05.823241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174072c-ad7e-4b20-89d6-cb2965d26849 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:05.833786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174072c-ad7e-4b20-89d6-cb2965d26849 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:05.840536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b929da-b239-455d-83bb-693ce77b641a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:05.841356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b929da-b239-455d-83bb-693ce77b641a service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:05.841591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b929da-b239-455d-83bb-693ce77b641a service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:05.842001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b929da-b239-455d-83bb-693ce77b641a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:05.909597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:05.931427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b929da-b239-455d-83bb-693ce77b641a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:05.932867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b929da-b239-455d-83bb-693ce77b641a service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:05.942920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 936/1868] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:15.727557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=38 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:15.938339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=10 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:15.946681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2beb2-1566-43ad-8430-91f3aac5c837 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:16.034175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2beb2-1566-43ad-8430-91f3aac5c837 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:16.041098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:16.042538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:16.042817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:16.043152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] PATH_INFO: `/v3/users/47c7c127f3364c91a9f264c3b8da6248` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:16.044314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c7c127f3364c91a9f264c3b8da6248)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:16.046292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:16.140667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c7c127f3364c91a9f264c3b8da6248'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.154040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c7c127f3364c91a9f264c3b8da6248'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.159464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c7c127f3364c91a9f264c3b8da6248'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.160775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c7c127f3364c91a9f264c3b8da6248'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.171255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c7c127f3364c91a9f264c3b8da6248'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.191252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 933/1869] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:15 2022] DELETE /identity/v3/users/47c7c127f3364c91a9f264c3b8da6248 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:16.195824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913590d-eaac-4ee9-ae6b-383847a884c3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:16.196189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913590d-eaac-4ee9-ae6b-383847a884c3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:16.196530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913590d-eaac-4ee9-ae6b-383847a884c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:16.217759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-4913590d-eaac-4ee9-ae6b-383847a884c3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:16.219108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4913590d-eaac-4ee9-ae6b-383847a884c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:16.280781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 937/1870] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:16.290935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:16.293156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:16.350539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a42bd-19b9-46b9-b881-23e09bae2d40 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:16.355153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108376b-cca9-4121-9245-ca6c31cfb8c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:16.355788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108376b-cca9-4121-9245-ca6c31cfb8c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:16.356159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108376b-cca9-4121-9245-ca6c31cfb8c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:16.356341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108376b-cca9-4121-9245-ca6c31cfb8c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:16.369380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108376b-cca9-4121-9245-ca6c31cfb8c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:16.371647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108376b-cca9-4121-9245-ca6c31cfb8c3 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:16.381847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 934/1871] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:16.661714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913590d-eaac-4ee9-ae6b-383847a884c3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:16.663271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4913590d-eaac-4ee9-ae6b-383847a884c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:16.676319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913590d-eaac-4ee9-ae6b-383847a884c3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:16.684081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:16.684737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:16.685019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:16.685293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] PATH_INFO: `/v3/projects/0ea97f8f601d4d059e1cd84fed8e938c` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:16.687310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea97f8f601d4d059e1cd84fed8e938c)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:16.689662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:16.736920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea97f8f601d4d059e1cd84fed8e938c', 'request_id': 'req-bebc7610-0abf-447e-8785-c2053b867ab5'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.744354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea97f8f601d4d059e1cd84fed8e938c', 'request_id': 'req-bebc7610-0abf-447e-8785-c2053b867ab5'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.750785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea97f8f601d4d059e1cd84fed8e938c', 'request_id': 'req-bebc7610-0abf-447e-8785-c2053b867ab5'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.761009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] The token cache is being invalidate because project 0ea97f8f601d4d059e1cd84fed8e938c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:16.761291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.761913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.763417 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 938/1872] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:16 2022] DELETE /identity/v3/projects/0ea97f8f601d4d059e1cd84fed8e938c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:16.767442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108376b-cca9-4121-9245-ca6c31cfb8c3 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:16.777619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108376b-cca9-4121-9245-ca6c31cfb8c3 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:16.796108 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:16.837977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:16.839155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:16.839441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:16.839713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] PATH_INFO: `/v3/users/6c7fe421e2d2426591c8a9acfd3bc160` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:16.840852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c7fe421e2d2426591c8a9acfd3bc160)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:16.842826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:16.892038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c7fe421e2d2426591c8a9acfd3bc160'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.901394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c7fe421e2d2426591c8a9acfd3bc160'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.902167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c7fe421e2d2426591c8a9acfd3bc160'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.902816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c7fe421e2d2426591c8a9acfd3bc160'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.910204 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-be74c2f1-b16f-4a4e-ab66-9cc68574d61a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c7fe421e2d2426591c8a9acfd3bc160'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:16.927898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 935/1873] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:16 2022] DELETE /identity/v3/users/6c7fe421e2d2426591c8a9acfd3bc160 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:17.318642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:17.320324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:17.344465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7610-0abf-447e-8785-c2053b867ab5 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:17.410121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:17.410772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.411088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.411369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] PATH_INFO: `/v3/projects/dc2c6f9df9bc4bf18ce7627b5605f1da` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.412691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2c6f9df9bc4bf18ce7627b5605f1da)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:17.414619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:17.448487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:17.469050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2c6f9df9bc4bf18ce7627b5605f1da', 'request_id': 'req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:17.476707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2c6f9df9bc4bf18ce7627b5605f1da', 'request_id': 'req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:17.483608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2c6f9df9bc4bf18ce7627b5605f1da', 'request_id': 'req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:17.496992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] The token cache is being invalidate because project dc2c6f9df9bc4bf18ce7627b5605f1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:17.497284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:17.498066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d9fb91c7-a88c-4d3c-a838-82add44c3b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:17.499747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 939/1874] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:17 2022] DELETE /identity/v3/projects/dc2c6f9df9bc4bf18ce7627b5605f1da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:17.504547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c2992-85d5-4ab0-b66e-127aa092ca21 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.504909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c2992-85d5-4ab0-b66e-127aa092ca21 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.505256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c2992-85d5-4ab0-b66e-127aa092ca21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.526534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-781c2992-85d5-4ab0-b66e-127aa092ca21 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:17.527938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-781c2992-85d5-4ab0-b66e-127aa092ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:17.596576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 936/1875] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:17.605507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fd5a1-06c6-4835-8739-b960649d02f3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.606028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fd5a1-06c6-4835-8739-b960649d02f3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.606451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fd5a1-06c6-4835-8739-b960649d02f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.643709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-776fd5a1-06c6-4835-8739-b960649d02f3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:17.645008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-776fd5a1-06c6-4835-8739-b960649d02f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:17.653003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 940/1876] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:17.657440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c2992-85d5-4ab0-b66e-127aa092ca21 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:17.669294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c2992-85d5-4ab0-b66e-127aa092ca21 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:17.673904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:17.674509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.674787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.675085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.675736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:17.676023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:17.677672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:17.692351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 937/1877] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:17.697201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fd5a1-06c6-4835-8739-b960649d02f3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:17.712540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fd5a1-06c6-4835-8739-b960649d02f3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:17.717480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fd245-ee54-498b-bff9-d17da7d37458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:17.718098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fd245-ee54-498b-bff9-d17da7d37458 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.718377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fd245-ee54-498b-bff9-d17da7d37458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.718642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fd245-ee54-498b-bff9-d17da7d37458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.719516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fd245-ee54-498b-bff9-d17da7d37458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:17.721748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fd245-ee54-498b-bff9-d17da7d37458 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:17.757119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 941/1878] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:17 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:17.762038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:17.772948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ec334-2e61-4eb2-94c1-428d4934bb81 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:17.804290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:17.841811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9194f90-6153-478b-bc50-b07b56710e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:17.842420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9194f90-6153-478b-bc50-b07b56710e68 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.842690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9194f90-6153-478b-bc50-b07b56710e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.842992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9194f90-6153-478b-bc50-b07b56710e68 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.843648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9194f90-6153-478b-bc50-b07b56710e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:17.845615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9194f90-6153-478b-bc50-b07b56710e68 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:17.870922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 938/1879] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:17 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:17.873628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fd245-ee54-498b-bff9-d17da7d37458 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:17.884745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fd245-ee54-498b-bff9-d17da7d37458 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:17.889449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bce2c-a6d8-4d6d-91fb-19b33db7d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:17.890131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bce2c-a6d8-4d6d-91fb-19b33db7d48d admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.890475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bce2c-a6d8-4d6d-91fb-19b33db7d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.890811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bce2c-a6d8-4d6d-91fb-19b33db7d48d admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.891545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bce2c-a6d8-4d6d-91fb-19b33db7d48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:17.893200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bce2c-a6d8-4d6d-91fb-19b33db7d48d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:17.903186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 942/1880] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:17.907420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9194f90-6153-478b-bc50-b07b56710e68 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:17.918511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9194f90-6153-478b-bc50-b07b56710e68 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:17.923372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5359dc5-6c03-4bdd-9420-c4e964a2a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:17.923916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5359dc5-6c03-4bdd-9420-c4e964a2a998 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.924197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5359dc5-6c03-4bdd-9420-c4e964a2a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.924470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5359dc5-6c03-4bdd-9420-c4e964a2a998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.925113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5359dc5-6c03-4bdd-9420-c4e964a2a998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:17.926703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5359dc5-6c03-4bdd-9420-c4e964a2a998 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:17.938505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 939/1881] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:17.943102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bce2c-a6d8-4d6d-91fb-19b33db7d48d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:17.953112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bce2c-a6d8-4d6d-91fb-19b33db7d48d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:17.957673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea1083-5e6d-4ea8-8eb1-a82a76ac54ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:17.958314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea1083-5e6d-4ea8-8eb1-a82a76ac54ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.958592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea1083-5e6d-4ea8-8eb1-a82a76ac54ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.958862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea1083-5e6d-4ea8-8eb1-a82a76ac54ec admin admin] PATH_INFO: `/v3/projects/0d267caa1bcf44e6a276ed4c9d5ee20f/users/c67b8a1fbbcc4a679080ddbbf8cd571a/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.972911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea1083-5e6d-4ea8-8eb1-a82a76ac54ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d267caa1bcf44e6a276ed4c9d5ee20f, user_id=c67b8a1fbbcc4a679080ddbbf8cd571a, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:17.975576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea1083-5e6d-4ea8-8eb1-a82a76ac54ec admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:17.990024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 943/1882] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:17 2022] PUT /identity/v3/projects/0d267caa1bcf44e6a276ed4c9d5ee20f/users/c67b8a1fbbcc4a679080ddbbf8cd571a/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:18.001032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e134c4-e997-48ac-9c15-33f99b39cdd1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.001318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e134c4-e997-48ac-9c15-33f99b39cdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.001590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e134c4-e997-48ac-9c15-33f99b39cdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.030532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e7e134c4-e997-48ac-9c15-33f99b39cdd1 None None] MFA Rules not processed for user `c67b8a1fbbcc4a679080ddbbf8cd571a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:18.031794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e7e134c4-e997-48ac-9c15-33f99b39cdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:18.096446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 940/1883] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:18.101090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea1083-5e6d-4ea8-8eb1-a82a76ac54ec admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:18.164778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea1083-5e6d-4ea8-8eb1-a82a76ac54ec admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:18.170431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fdb8c-d7a2-4a50-9af4-824540efab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:18.171071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fdb8c-d7a2-4a50-9af4-824540efab6f admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.171359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fdb8c-d7a2-4a50-9af4-824540efab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.171632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fdb8c-d7a2-4a50-9af4-824540efab6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.172282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fdb8c-d7a2-4a50-9af4-824540efab6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:18.174274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fdb8c-d7a2-4a50-9af4-824540efab6f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:18.202393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 944/1884] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:18 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:18.206658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e134c4-e997-48ac-9c15-33f99b39cdd1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:18.217930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e134c4-e997-48ac-9c15-33f99b39cdd1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:18.272564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2035f7-048e-44d8-ad22-c3c071547cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:18.273179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2035f7-048e-44d8-ad22-c3c071547cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.273617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2035f7-048e-44d8-ad22-c3c071547cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.273909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2035f7-048e-44d8-ad22-c3c071547cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.274533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2035f7-048e-44d8-ad22-c3c071547cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:18.277004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2035f7-048e-44d8-ad22-c3c071547cb4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:18.297220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 941/1885] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:18 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:18.301191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fdb8c-d7a2-4a50-9af4-824540efab6f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:18.312253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fdb8c-d7a2-4a50-9af4-824540efab6f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:18.316865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccc48d-5c42-4e4f-a4ad-8093b72dccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:18.317460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc48d-5c42-4e4f-a4ad-8093b72dccb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.317740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc48d-5c42-4e4f-a4ad-8093b72dccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.318010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc48d-5c42-4e4f-a4ad-8093b72dccb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.318645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccc48d-5c42-4e4f-a4ad-8093b72dccb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:18.320278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccc48d-5c42-4e4f-a4ad-8093b72dccb4 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:18.331128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 945/1886] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:18.334836 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2035f7-048e-44d8-ad22-c3c071547cb4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:18.345390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2035f7-048e-44d8-ad22-c3c071547cb4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:18.350159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b49961-463f-4b88-8138-cd8bc8ba9fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:18.350811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b49961-463f-4b88-8138-cd8bc8ba9fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.351119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b49961-463f-4b88-8138-cd8bc8ba9fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.351391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b49961-463f-4b88-8138-cd8bc8ba9fac admin admin] PATH_INFO: `/v3/projects/99e6328c4f52439f9aa1a1a01ad3e6bd/users/cde94cfbac364fb59e1d31bd60a3c2cd/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.366542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b49961-463f-4b88-8138-cd8bc8ba9fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e6328c4f52439f9aa1a1a01ad3e6bd, user_id=cde94cfbac364fb59e1d31bd60a3c2cd, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:18.369208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b49961-463f-4b88-8138-cd8bc8ba9fac admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:18.381787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 942/1887] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:18 2022] PUT /identity/v3/projects/99e6328c4f52439f9aa1a1a01ad3e6bd/users/cde94cfbac364fb59e1d31bd60a3c2cd/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:18.385993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccc48d-5c42-4e4f-a4ad-8093b72dccb4 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:18.395589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccc48d-5c42-4e4f-a4ad-8093b72dccb4 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:18.415924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:18.455742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415d914-e386-47fe-8e08-b6f06e20451e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:18.456279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415d914-e386-47fe-8e08-b6f06e20451e admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.456575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415d914-e386-47fe-8e08-b6f06e20451e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.456831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415d914-e386-47fe-8e08-b6f06e20451e admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.457539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415d914-e386-47fe-8e08-b6f06e20451e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:18.459453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415d914-e386-47fe-8e08-b6f06e20451e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:18.473757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 946/1888] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:18.475071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b49961-463f-4b88-8138-cd8bc8ba9fac admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:18.481281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:18.486848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b49961-463f-4b88-8138-cd8bc8ba9fac admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:18.492351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:18.493000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.493282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.493560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] PATH_INFO: `/v3/projects/99e6328c4f52439f9aa1a1a01ad3e6bd/users/cde94cfbac364fb59e1d31bd60a3c2cd/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.495821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e6328c4f52439f9aa1a1a01ad3e6bd, user_id=cde94cfbac364fb59e1d31bd60a3c2cd, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:18.498621 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:18.508634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 943/1889] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:18 2022] PUT /identity/v3/projects/99e6328c4f52439f9aa1a1a01ad3e6bd/users/cde94cfbac364fb59e1d31bd60a3c2cd/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:18.520165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593eed53-9452-47de-a35b-57b735e0c31b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.520453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593eed53-9452-47de-a35b-57b735e0c31b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.520726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593eed53-9452-47de-a35b-57b735e0c31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.548363 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-593eed53-9452-47de-a35b-57b735e0c31b None None] MFA Rules not processed for user `cde94cfbac364fb59e1d31bd60a3c2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:18.549555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-593eed53-9452-47de-a35b-57b735e0c31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:18.616715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 947/1890] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:18.629746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:18.631648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:18.708225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f79f91-cb65-41ff-b85d-ee5f2a58f6bc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:18.713893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc8f56-2665-4cba-95d2-13bcdf7e9b1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:18.714510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8f56-2665-4cba-95d2-13bcdf7e9b1f service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.714793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8f56-2665-4cba-95d2-13bcdf7e9b1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.715103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8f56-2665-4cba-95d2-13bcdf7e9b1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.725121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc8f56-2665-4cba-95d2-13bcdf7e9b1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:18.726546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc8f56-2665-4cba-95d2-13bcdf7e9b1f service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:18.736372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 944/1891] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:19.069008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593eed53-9452-47de-a35b-57b735e0c31b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:19.078729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593eed53-9452-47de-a35b-57b735e0c31b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:19.083243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cde94cfbac364fb59e1d31bd60a3c2cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6328c4f52439f9aa1a1a01ad3e6bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:19.083837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:19.084121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:19.084391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:19.085028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:19.085385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:19.086990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:19.098424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 948/1892] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:19.102810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc8f56-2665-4cba-95d2-13bcdf7e9b1f service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:19.112035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc8f56-2665-4cba-95d2-13bcdf7e9b1f service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:19.116839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5b73a-28e2-4597-bd2e-11bf6357d60d tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cde94cfbac364fb59e1d31bd60a3c2cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6328c4f52439f9aa1a1a01ad3e6bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:19.117445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5b73a-28e2-4597-bd2e-11bf6357d60d tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:19.117732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5b73a-28e2-4597-bd2e-11bf6357d60d tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:19.118008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5b73a-28e2-4597-bd2e-11bf6357d60d tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:19.118637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5b73a-28e2-4597-bd2e-11bf6357d60d tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:19.121155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5b73a-28e2-4597-bd2e-11bf6357d60d tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:19.149592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 945/1893] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:19 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:19.153875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:19.164731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf1936-a2d5-4fa0-b3e8-a3abcdb8d04b tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:19.229655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cde94cfbac364fb59e1d31bd60a3c2cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6328c4f52439f9aa1a1a01ad3e6bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:19.230356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:19.230744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:19.231337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:19.231746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:19.232045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:19.233619 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:19.246791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 949/1894] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:19.286443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5b73a-28e2-4597-bd2e-11bf6357d60d tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:19.296853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5b73a-28e2-4597-bd2e-11bf6357d60d tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:19.301844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cde94cfbac364fb59e1d31bd60a3c2cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6328c4f52439f9aa1a1a01ad3e6bd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:19.302566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:19.302996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:19.303159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] PATH_INFO: `/v3/projects/307bd262f307400d80a1013fd27cc5a5` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:19.304562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Authorizing `identity:delete_project(project_id=307bd262f307400d80a1013fd27cc5a5)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:19.306477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:19.360812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307bd262f307400d80a1013fd27cc5a5', 'request_id': 'req-8ec21fed-9609-4d06-9eec-65deaf54d2e5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.368723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307bd262f307400d80a1013fd27cc5a5', 'request_id': 'req-8ec21fed-9609-4d06-9eec-65deaf54d2e5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.379528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307bd262f307400d80a1013fd27cc5a5', 'request_id': 'req-8ec21fed-9609-4d06-9eec-65deaf54d2e5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.390807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] The token cache is being invalidate because project 307bd262f307400d80a1013fd27cc5a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:19.391145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.391877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ec21fed-9609-4d06-9eec-65deaf54d2e5 tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.393400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 946/1895] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:19 2022] DELETE /identity/v3/projects/307bd262f307400d80a1013fd27cc5a5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:19.593960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:19.595636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:19.626976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:19.660795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fea55-42f6-4249-8b89-df00fc435e1a tempest-VolumeQuotaClassesTest-1389855670 tempest-VolumeQuotaClassesTest-1389855670-project-admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:19.720330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:19.721034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:19.721281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:19.721556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] PATH_INFO: `/v3/users/c67b8a1fbbcc4a679080ddbbf8cd571a` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:19.725766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c67b8a1fbbcc4a679080ddbbf8cd571a)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:19.727778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:19.783699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c67b8a1fbbcc4a679080ddbbf8cd571a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.790918 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c67b8a1fbbcc4a679080ddbbf8cd571a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.792194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c67b8a1fbbcc4a679080ddbbf8cd571a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.792845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c67b8a1fbbcc4a679080ddbbf8cd571a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.802549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c67b8a1fbbcc4a679080ddbbf8cd571a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:19.821630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 950/1896] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:19 2022] DELETE /identity/v3/users/c67b8a1fbbcc4a679080ddbbf8cd571a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:19.826078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578cb8a-9a21-409d-957a-7d62eac379ac None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:19.826771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578cb8a-9a21-409d-957a-7d62eac379ac None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:19.827244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578cb8a-9a21-409d-957a-7d62eac379ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:19.851261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8578cb8a-9a21-409d-957a-7d62eac379ac None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:19.852925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8578cb8a-9a21-409d-957a-7d62eac379ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:19.921411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:19.935788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 947/1897] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:19.946717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:19.948383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:20.012367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4118e0-18d6-4fa8-a7da-c4493971cdb2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:20.016045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1ace3-eb1e-4e14-8cac-f036d1afbc76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:20.016722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1ace3-eb1e-4e14-8cac-f036d1afbc76 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:20.017071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1ace3-eb1e-4e14-8cac-f036d1afbc76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:20.017530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1ace3-eb1e-4e14-8cac-f036d1afbc76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:20.024885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:20.028561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1ace3-eb1e-4e14-8cac-f036d1afbc76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:20.030227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1ace3-eb1e-4e14-8cac-f036d1afbc76 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:20.043251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 951/1898] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:20.653056 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578cb8a-9a21-409d-957a-7d62eac379ac None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:20.654709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8578cb8a-9a21-409d-957a-7d62eac379ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:20.667689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578cb8a-9a21-409d-957a-7d62eac379ac None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:20.672795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:20.673427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:20.673710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:20.673985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] PATH_INFO: `/v3/projects/0d267caa1bcf44e6a276ed4c9d5ee20f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:20.675225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d267caa1bcf44e6a276ed4c9d5ee20f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:20.677500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:20.744064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d267caa1bcf44e6a276ed4c9d5ee20f', 'request_id': 'req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.755048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d267caa1bcf44e6a276ed4c9d5ee20f', 'request_id': 'req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.767673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d267caa1bcf44e6a276ed4c9d5ee20f', 'request_id': 'req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.783796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] The token cache is being invalidate because project 0d267caa1bcf44e6a276ed4c9d5ee20f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:20.784129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.784951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.786754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 948/1899] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:20 2022] DELETE /identity/v3/projects/0d267caa1bcf44e6a276ed4c9d5ee20f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:20.794008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1ace3-eb1e-4e14-8cac-f036d1afbc76 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:20.817234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1ace3-eb1e-4e14-8cac-f036d1afbc76 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:20.907414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:20.908165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:20.908584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:20.908861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] PATH_INFO: `/v3/users/cde94cfbac364fb59e1d31bd60a3c2cd` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:20.910330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cde94cfbac364fb59e1d31bd60a3c2cd)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:20.912449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:20.975703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cde94cfbac364fb59e1d31bd60a3c2cd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.984128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cde94cfbac364fb59e1d31bd60a3c2cd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.984822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cde94cfbac364fb59e1d31bd60a3c2cd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.985431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cde94cfbac364fb59e1d31bd60a3c2cd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:20.993199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-cf32b243-6832-4ede-9097-2703e7d9aceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cde94cfbac364fb59e1d31bd60a3c2cd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:21.012113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 952/1900] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:20 2022] DELETE /identity/v3/users/cde94cfbac364fb59e1d31bd60a3c2cd => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:21.285041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:21.286451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:21.303397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70aa9f-9cd6-4aee-a52d-9021a114ff18 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:21.361298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:21.361959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.362239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.362514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] PATH_INFO: `/v3/projects/99e6328c4f52439f9aa1a1a01ad3e6bd` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.363655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e6328c4f52439f9aa1a1a01ad3e6bd)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:21.365550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:21.418101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e6328c4f52439f9aa1a1a01ad3e6bd', 'request_id': 'req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:21.425748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e6328c4f52439f9aa1a1a01ad3e6bd', 'request_id': 'req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:21.433464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e6328c4f52439f9aa1a1a01ad3e6bd', 'request_id': 'req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:21.444063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] The token cache is being invalidate because project 99e6328c4f52439f9aa1a1a01ad3e6bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:21.444354 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:21.445004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-97ddb7f5-b935-487d-bada-e3b5e0e64c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:21.446847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 949/1901] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:21 2022] DELETE /identity/v3/projects/99e6328c4f52439f9aa1a1a01ad3e6bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:21.452427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889bc1a-6629-46aa-b882-50989d9316f3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.452723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889bc1a-6629-46aa-b882-50989d9316f3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.453002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889bc1a-6629-46aa-b882-50989d9316f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.473158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a889bc1a-6629-46aa-b882-50989d9316f3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:21.474288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a889bc1a-6629-46aa-b882-50989d9316f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:21.535560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 953/1902] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:21.543425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259ae2e-1931-4e21-9945-e8a88ba0ec22 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.543711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259ae2e-1931-4e21-9945-e8a88ba0ec22 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.544021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259ae2e-1931-4e21-9945-e8a88ba0ec22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.559761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:21.565530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6259ae2e-1931-4e21-9945-e8a88ba0ec22 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:21.567368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6259ae2e-1931-4e21-9945-e8a88ba0ec22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:21.576254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 950/1903] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:21.580523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889bc1a-6629-46aa-b882-50989d9316f3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:21.590108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889bc1a-6629-46aa-b882-50989d9316f3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:21.594684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:21.595311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.595585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.595853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.596489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:21.596789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:21.598368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:21.610556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 954/1904] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:21.614818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259ae2e-1931-4e21-9945-e8a88ba0ec22 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:21.624597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259ae2e-1931-4e21-9945-e8a88ba0ec22 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:21.629204 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87126b47-989d-4aef-9768-f4266d46d2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:21.629795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87126b47-989d-4aef-9768-f4266d46d2bc admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.630075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87126b47-989d-4aef-9768-f4266d46d2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.630344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87126b47-989d-4aef-9768-f4266d46d2bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.631005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87126b47-989d-4aef-9768-f4266d46d2bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:21.632910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87126b47-989d-4aef-9768-f4266d46d2bc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:21.660541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 951/1905] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:21 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:21.664716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:21.670598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:21.676848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb71f23-2f5d-4494-94b1-ba255e96429f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:21.733539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500acd13-b7c7-4b73-b105-8d3bbf70be45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:21.734055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500acd13-b7c7-4b73-b105-8d3bbf70be45 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.734330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500acd13-b7c7-4b73-b105-8d3bbf70be45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.734598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500acd13-b7c7-4b73-b105-8d3bbf70be45 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.735260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500acd13-b7c7-4b73-b105-8d3bbf70be45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:21.737318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500acd13-b7c7-4b73-b105-8d3bbf70be45 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:21.759305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 955/1906] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:21 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:21.763607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87126b47-989d-4aef-9768-f4266d46d2bc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:21.774411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87126b47-989d-4aef-9768-f4266d46d2bc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:21.779085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e461652-9d30-47f2-ae11-31cc03cbba10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:21.779690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461652-9d30-47f2-ae11-31cc03cbba10 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.779966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461652-9d30-47f2-ae11-31cc03cbba10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.780293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461652-9d30-47f2-ae11-31cc03cbba10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.780950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e461652-9d30-47f2-ae11-31cc03cbba10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:21.782692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e461652-9d30-47f2-ae11-31cc03cbba10 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:21.793405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 952/1907] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:21.797442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500acd13-b7c7-4b73-b105-8d3bbf70be45 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:21.808795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500acd13-b7c7-4b73-b105-8d3bbf70be45 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:21.813496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a895a5f-ad64-4e3f-88f7-962c1583df9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:21.814108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a895a5f-ad64-4e3f-88f7-962c1583df9d admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.814390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a895a5f-ad64-4e3f-88f7-962c1583df9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.814669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a895a5f-ad64-4e3f-88f7-962c1583df9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.815360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a895a5f-ad64-4e3f-88f7-962c1583df9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:21.816954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a895a5f-ad64-4e3f-88f7-962c1583df9d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:21.827901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 956/1908] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:21.832154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e461652-9d30-47f2-ae11-31cc03cbba10 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:21.842554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e461652-9d30-47f2-ae11-31cc03cbba10 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:21.847188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8106d-3d80-4ebb-9c7d-ce2f3db745a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:21.847845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8106d-3d80-4ebb-9c7d-ce2f3db745a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.848129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8106d-3d80-4ebb-9c7d-ce2f3db745a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.848405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8106d-3d80-4ebb-9c7d-ce2f3db745a4 admin admin] PATH_INFO: `/v3/projects/37200ae5335e436f8d25d17f00b9abf9/users/25b8e8ac36744d02b59e09b7750c103c/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.863563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8106d-3d80-4ebb-9c7d-ce2f3db745a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37200ae5335e436f8d25d17f00b9abf9, user_id=25b8e8ac36744d02b59e09b7750c103c, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:21.866219 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8106d-3d80-4ebb-9c7d-ce2f3db745a4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:21.880010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 953/1909] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:21 2022] PUT /identity/v3/projects/37200ae5335e436f8d25d17f00b9abf9/users/25b8e8ac36744d02b59e09b7750c103c/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:21.891712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ecf3c-a8d5-4f4b-86ad-120a292e0f85 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.891996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ecf3c-a8d5-4f4b-86ad-120a292e0f85 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.892258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ecf3c-a8d5-4f4b-86ad-120a292e0f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.924154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f94ecf3c-a8d5-4f4b-86ad-120a292e0f85 None None] MFA Rules not processed for user `25b8e8ac36744d02b59e09b7750c103c`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:21.925358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f94ecf3c-a8d5-4f4b-86ad-120a292e0f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:22.002787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 957/1910] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:21 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:22.014701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8106d-3d80-4ebb-9c7d-ce2f3db745a4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:22.091456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8106d-3d80-4ebb-9c7d-ce2f3db745a4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:22.094284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce0300-359c-458c-8e0d-d012afd79b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:22.095143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce0300-359c-458c-8e0d-d012afd79b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.095348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce0300-359c-458c-8e0d-d012afd79b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.095648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce0300-359c-458c-8e0d-d012afd79b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.096420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce0300-359c-458c-8e0d-d012afd79b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:22.098442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce0300-359c-458c-8e0d-d012afd79b2f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:22.149539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 954/1911] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:22 2022] POST /identity/v3/projects => generated 401 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:22.154180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ecf3c-a8d5-4f4b-86ad-120a292e0f85 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:22.172185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ecf3c-a8d5-4f4b-86ad-120a292e0f85 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:22.253335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0584d6-894a-4a6d-af95-65683becc760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:22.254516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0584d6-894a-4a6d-af95-65683becc760 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.255024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0584d6-894a-4a6d-af95-65683becc760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.255447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0584d6-894a-4a6d-af95-65683becc760 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.256259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0584d6-894a-4a6d-af95-65683becc760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:22.258437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0584d6-894a-4a6d-af95-65683becc760 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:22.277094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:22.286004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 958/1912] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:22 2022] POST /identity/v3/users => generated 307 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:22.286997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce0300-359c-458c-8e0d-d012afd79b2f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:22.298626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce0300-359c-458c-8e0d-d012afd79b2f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:22.303230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c4ccb-7bc3-4da0-90f2-78807337d54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:22.304714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c4ccb-7bc3-4da0-90f2-78807337d54c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.304994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c4ccb-7bc3-4da0-90f2-78807337d54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.305269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c4ccb-7bc3-4da0-90f2-78807337d54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.305936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c4ccb-7bc3-4da0-90f2-78807337d54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:22.307607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c4ccb-7bc3-4da0-90f2-78807337d54c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:22.320103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 955/1913] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:22.324427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0584d6-894a-4a6d-af95-65683becc760 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:22.334317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0584d6-894a-4a6d-af95-65683becc760 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:22.339547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a56ce2-6c0d-42de-9cb0-f4dbb95a8264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:22.340212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a56ce2-6c0d-42de-9cb0-f4dbb95a8264 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.340494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a56ce2-6c0d-42de-9cb0-f4dbb95a8264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.340769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a56ce2-6c0d-42de-9cb0-f4dbb95a8264 admin admin] PATH_INFO: `/v3/projects/6f00dfd834a246eaa8ffe4cbd703a785/users/64dfeedb13b449dd9a15c825f31c1e53/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.356949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a56ce2-6c0d-42de-9cb0-f4dbb95a8264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f00dfd834a246eaa8ffe4cbd703a785, user_id=64dfeedb13b449dd9a15c825f31c1e53, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:22.359640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a56ce2-6c0d-42de-9cb0-f4dbb95a8264 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:22.373506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 959/1914] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:22 2022] PUT /identity/v3/projects/6f00dfd834a246eaa8ffe4cbd703a785/users/64dfeedb13b449dd9a15c825f31c1e53/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:22.379966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c4ccb-7bc3-4da0-90f2-78807337d54c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:22.392206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c4ccb-7bc3-4da0-90f2-78807337d54c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:22.452833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c2603-152c-4050-9476-31a2ee12e24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:22.453460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c2603-152c-4050-9476-31a2ee12e24d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.453734 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c2603-152c-4050-9476-31a2ee12e24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.454024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c2603-152c-4050-9476-31a2ee12e24d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.454661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c2603-152c-4050-9476-31a2ee12e24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:22.456545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c2603-152c-4050-9476-31a2ee12e24d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:22.467679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 956/1915] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:22.471693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a56ce2-6c0d-42de-9cb0-f4dbb95a8264 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:22.482155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a56ce2-6c0d-42de-9cb0-f4dbb95a8264 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:22.486865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:22.487556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.487839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.488113 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] PATH_INFO: `/v3/projects/6f00dfd834a246eaa8ffe4cbd703a785/users/64dfeedb13b449dd9a15c825f31c1e53/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.489978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f00dfd834a246eaa8ffe4cbd703a785, user_id=64dfeedb13b449dd9a15c825f31c1e53, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:22.492636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:22.502973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 960/1916] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:22 2022] PUT /identity/v3/projects/6f00dfd834a246eaa8ffe4cbd703a785/users/64dfeedb13b449dd9a15c825f31c1e53/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:22.514064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc944-1b18-4bb2-8ec4-c679b068b6f3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.514289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc944-1b18-4bb2-8ec4-c679b068b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.514754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc944-1b18-4bb2-8ec4-c679b068b6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.540432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:22.543862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-69fdc944-1b18-4bb2-8ec4-c679b068b6f3 None None] MFA Rules not processed for user `64dfeedb13b449dd9a15c825f31c1e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:22.546056 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-69fdc944-1b18-4bb2-8ec4-c679b068b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:22.615595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 957/1917] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:22 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:22.626863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:22.628612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:22.687670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41858de-8229-46d9-bee9-2ab3315e45c8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:22.691335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506badfd-786b-4c1c-adfe-963a57ea8e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:22.691944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506badfd-786b-4c1c-adfe-963a57ea8e8e service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.692224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506badfd-786b-4c1c-adfe-963a57ea8e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.692497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506badfd-786b-4c1c-adfe-963a57ea8e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.702230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506badfd-786b-4c1c-adfe-963a57ea8e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:22.703916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506badfd-786b-4c1c-adfe-963a57ea8e8e service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:22.713901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 961/1918] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:22 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:22.791037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc944-1b18-4bb2-8ec4-c679b068b6f3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:22.801189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc944-1b18-4bb2-8ec4-c679b068b6f3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:22.805115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d48fe-b087-4384-bd2e-d539013e317a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:22.805728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d48fe-b087-4384-bd2e-d539013e317a service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:22.806008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d48fe-b087-4384-bd2e-d539013e317a service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:22.806282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d48fe-b087-4384-bd2e-d539013e317a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:22.863586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d48fe-b087-4384-bd2e-d539013e317a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:22.865031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d48fe-b087-4384-bd2e-d539013e317a service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:22.874941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 958/1919] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:22 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:32.389838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506badfd-786b-4c1c-adfe-963a57ea8e8e service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:32.460436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506badfd-786b-4c1c-adfe-963a57ea8e8e service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:32.466018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:32.466725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:32.466994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:32.467274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] PATH_INFO: `/v3/users/25b8e8ac36744d02b59e09b7750c103c` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:32.468471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b8e8ac36744d02b59e09b7750c103c)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:32.470409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:32.516238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b8e8ac36744d02b59e09b7750c103c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:32.523323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b8e8ac36744d02b59e09b7750c103c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:32.523976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b8e8ac36744d02b59e09b7750c103c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:32.524592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b8e8ac36744d02b59e09b7750c103c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:32.533038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b8e8ac36744d02b59e09b7750c103c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:32.537545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:32.553281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 962/1920] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:32 2022] DELETE /identity/v3/users/25b8e8ac36744d02b59e09b7750c103c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:32.558177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62be0f-afa8-404e-a899-1302ac6f7cb2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:32.558630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62be0f-afa8-404e-a899-1302ac6f7cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:32.558737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62be0f-afa8-404e-a899-1302ac6f7cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:32.577753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6f62be0f-afa8-404e-a899-1302ac6f7cb2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:32.578995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6f62be0f-afa8-404e-a899-1302ac6f7cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:32.638452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 959/1921] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:32.648180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:32.649690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:32.706703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f52af5-0cac-43cf-b244-c2b808ee63ea admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:32.710327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afe2fd-fbf0-48c8-b79f-601b64a3461c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:32.711009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afe2fd-fbf0-48c8-b79f-601b64a3461c service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:32.711238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afe2fd-fbf0-48c8-b79f-601b64a3461c service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:32.711511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afe2fd-fbf0-48c8-b79f-601b64a3461c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:32.720955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afe2fd-fbf0-48c8-b79f-601b64a3461c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:32.722370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afe2fd-fbf0-48c8-b79f-601b64a3461c service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:32.732238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 963/1922] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:32.995155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62be0f-afa8-404e-a899-1302ac6f7cb2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:32.996629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6f62be0f-afa8-404e-a899-1302ac6f7cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:33.020957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62be0f-afa8-404e-a899-1302ac6f7cb2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:33.028500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:33.029119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:33.029462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:33.029884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] PATH_INFO: `/v3/projects/37200ae5335e436f8d25d17f00b9abf9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:33.031122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37200ae5335e436f8d25d17f00b9abf9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:33.033134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:33.070527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:33.085039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37200ae5335e436f8d25d17f00b9abf9', 'request_id': 'req-7f960367-4abc-416f-be73-64169710cda7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.092684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37200ae5335e436f8d25d17f00b9abf9', 'request_id': 'req-7f960367-4abc-416f-be73-64169710cda7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.099706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37200ae5335e436f8d25d17f00b9abf9', 'request_id': 'req-7f960367-4abc-416f-be73-64169710cda7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.110571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] The token cache is being invalidate because project 37200ae5335e436f8d25d17f00b9abf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:33.110858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.111704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.113493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 960/1923] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:32 2022] DELETE /identity/v3/projects/37200ae5335e436f8d25d17f00b9abf9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:33.117495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afe2fd-fbf0-48c8-b79f-601b64a3461c service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:33.127637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afe2fd-fbf0-48c8-b79f-601b64a3461c service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:33.189648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:33.190275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:33.190549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:33.190819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] PATH_INFO: `/v3/users/64dfeedb13b449dd9a15c825f31c1e53` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:33.191942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64dfeedb13b449dd9a15c825f31c1e53)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:33.193941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:33.241337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64dfeedb13b449dd9a15c825f31c1e53'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.248905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64dfeedb13b449dd9a15c825f31c1e53'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.249555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64dfeedb13b449dd9a15c825f31c1e53'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.250166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64dfeedb13b449dd9a15c825f31c1e53'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.257814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a072a2b8-e20c-44e5-8779-cc51bbdaf7e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64dfeedb13b449dd9a15c825f31c1e53'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.278530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 964/1924] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:33 2022] DELETE /identity/v3/users/64dfeedb13b449dd9a15c825f31c1e53 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:33.595325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:33.598025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:33.631466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f960367-4abc-416f-be73-64169710cda7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:33.690742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:33.691409 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:33.691690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:33.691963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] PATH_INFO: `/v3/projects/6f00dfd834a246eaa8ffe4cbd703a785` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:33.693138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f00dfd834a246eaa8ffe4cbd703a785)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:33.695080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:33.743011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f00dfd834a246eaa8ffe4cbd703a785', 'request_id': 'req-c689170b-3450-4138-b01c-177fbcb1c62a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.751294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f00dfd834a246eaa8ffe4cbd703a785', 'request_id': 'req-c689170b-3450-4138-b01c-177fbcb1c62a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.757966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f00dfd834a246eaa8ffe4cbd703a785', 'request_id': 'req-c689170b-3450-4138-b01c-177fbcb1c62a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.768757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] The token cache is being invalidate because project 6f00dfd834a246eaa8ffe4cbd703a785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:33.769039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.769723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c689170b-3450-4138-b01c-177fbcb1c62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:33.771240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 961/1925] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:33 2022] DELETE /identity/v3/projects/6f00dfd834a246eaa8ffe4cbd703a785 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:33.776474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce182e-348b-4e73-8663-f49a46d0debb None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:33.776922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce182e-348b-4e73-8663-f49a46d0debb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:33.777028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce182e-348b-4e73-8663-f49a46d0debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:33.790290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:33.797246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f4ce182e-348b-4e73-8663-f49a46d0debb None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:33.799214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f4ce182e-348b-4e73-8663-f49a46d0debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:33.857208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 965/1926] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:33.865051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9515ac1-2625-4099-9289-5c456a4c3ef8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:33.865345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9515ac1-2625-4099-9289-5c456a4c3ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:33.865616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9515ac1-2625-4099-9289-5c456a4c3ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:33.885662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a9515ac1-2625-4099-9289-5c456a4c3ef8 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:33.886840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a9515ac1-2625-4099-9289-5c456a4c3ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:33.894590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 962/1927] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:33.898734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce182e-348b-4e73-8663-f49a46d0debb None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:33.908146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce182e-348b-4e73-8663-f49a46d0debb None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:33.912541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:33.913135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:33.913410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:33.913676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:33.914316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:33.914613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:33.916808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:33.928375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 966/1928] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:33.933370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9515ac1-2625-4099-9289-5c456a4c3ef8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:33.942395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9515ac1-2625-4099-9289-5c456a4c3ef8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:33.946913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153cdc2-d45a-470a-8e6b-2d92728a321f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:33.947540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cdc2-d45a-470a-8e6b-2d92728a321f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:33.947822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cdc2-d45a-470a-8e6b-2d92728a321f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:33.948093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cdc2-d45a-470a-8e6b-2d92728a321f admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:33.948723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153cdc2-d45a-470a-8e6b-2d92728a321f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:33.950595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153cdc2-d45a-470a-8e6b-2d92728a321f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:33.977723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 963/1929] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:33 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:33.982611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:33.992317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667cdae-8497-4f0a-9a30-c9bbf9e98ca8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.048536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a51082-3d4b-42e5-bd73-6875f5428b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.049063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a51082-3d4b-42e5-bd73-6875f5428b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.049341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a51082-3d4b-42e5-bd73-6875f5428b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.049609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a51082-3d4b-42e5-bd73-6875f5428b72 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.050236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a51082-3d4b-42e5-bd73-6875f5428b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.052297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a51082-3d4b-42e5-bd73-6875f5428b72 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.073186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 967/1930] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:33 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:34.077313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153cdc2-d45a-470a-8e6b-2d92728a321f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.088478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153cdc2-d45a-470a-8e6b-2d92728a321f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.093659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.094278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.094546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.094831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.095498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.097416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.111548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 964/1931] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:34.112718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a51082-3d4b-42e5-bd73-6875f5428b72 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.123370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a51082-3d4b-42e5-bd73-6875f5428b72 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.128067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234233-a163-4045-af7e-99bbbd2d5172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.128679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234233-a163-4045-af7e-99bbbd2d5172 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.128999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234233-a163-4045-af7e-99bbbd2d5172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.129230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234233-a163-4045-af7e-99bbbd2d5172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.130050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b234233-a163-4045-af7e-99bbbd2d5172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.131677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b234233-a163-4045-af7e-99bbbd2d5172 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.132223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.134139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.145618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 968/1932] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:34.146929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234233-a163-4045-af7e-99bbbd2d5172 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.149443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefebd8e-c83a-470b-a48e-741ecf4bfdac admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.156417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.157062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.157347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.157626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] PATH_INFO: `/v3/users/32fe3636c0b4432fbb453892e68ac280` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.158092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234233-a163-4045-af7e-99bbbd2d5172 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.158876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fe3636c0b4432fbb453892e68ac280)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.161253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.164012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cf3de-a770-4f64-a0f5-48021d1bedc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.165075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cf3de-a770-4f64-a0f5-48021d1bedc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.165389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cf3de-a770-4f64-a0f5-48021d1bedc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.165703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cf3de-a770-4f64-a0f5-48021d1bedc7 admin admin] PATH_INFO: `/v3/projects/65f225fe4d0c46e7a00ccf16dd5fa88a/users/3dfafdac717849c2a8dae38c76eca624/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.167978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:34.185531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cf3de-a770-4f64-a0f5-48021d1bedc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f225fe4d0c46e7a00ccf16dd5fa88a, user_id=3dfafdac717849c2a8dae38c76eca624, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.188243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cf3de-a770-4f64-a0f5-48021d1bedc7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.202785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 969/1933] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:34 2022] PUT /identity/v3/projects/65f225fe4d0c46e7a00ccf16dd5fa88a/users/3dfafdac717849c2a8dae38c76eca624/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:34.214346 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070bcb-50e5-4f79-92c0-4e21c068a8f3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.214634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070bcb-50e5-4f79-92c0-4e21c068a8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.214928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070bcb-50e5-4f79-92c0-4e21c068a8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.215579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fe3636c0b4432fbb453892e68ac280'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:34.223156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fe3636c0b4432fbb453892e68ac280'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:34.224040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fe3636c0b4432fbb453892e68ac280'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:34.224733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fe3636c0b4432fbb453892e68ac280'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:34.232581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-12dce5e6-ff6a-4838-8e83-7afb2a053a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fe3636c0b4432fbb453892e68ac280'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:34.243508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-29070bcb-50e5-4f79-92c0-4e21c068a8f3 None None] MFA Rules not processed for user `3dfafdac717849c2a8dae38c76eca624`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:34.244713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-29070bcb-50e5-4f79-92c0-4e21c068a8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.251406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 965/1934] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:34 2022] DELETE /identity/v3/users/32fe3636c0b4432fbb453892e68ac280 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:34.255724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3e4b-a15f-4d57-97ef-da2870330f46 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.256072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3e4b-a15f-4d57-97ef-da2870330f46 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.256357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3e4b-a15f-4d57-97ef-da2870330f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.279863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-829b3e4b-a15f-4d57-97ef-da2870330f46 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:34.281228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-829b3e4b-a15f-4d57-97ef-da2870330f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.287929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:34.317586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 970/1935] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:34.322644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29070bcb-50e5-4f79-92c0-4e21c068a8f3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.324552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-29070bcb-50e5-4f79-92c0-4e21c068a8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.344385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 966/1936] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:34.353814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b3e4b-a15f-4d57-97ef-da2870330f46 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.355553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-829b3e4b-a15f-4d57-97ef-da2870330f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.392428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29070bcb-50e5-4f79-92c0-4e21c068a8f3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.396885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db6e5-e15a-495c-ad64-d28fb8e71d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.397487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db6e5-e15a-495c-ad64-d28fb8e71d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.397769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db6e5-e15a-495c-ad64-d28fb8e71d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.398035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db6e5-e15a-495c-ad64-d28fb8e71d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.398647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440db6e5-e15a-495c-ad64-d28fb8e71d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.401218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440db6e5-e15a-495c-ad64-d28fb8e71d00 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.415921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b3e4b-a15f-4d57-97ef-da2870330f46 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.420987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9c325-8cfe-45b8-936a-7e940bc26644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.421647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9c325-8cfe-45b8-936a-7e940bc26644 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.421956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9c325-8cfe-45b8-936a-7e940bc26644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.422229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9c325-8cfe-45b8-936a-7e940bc26644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.430373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 971/1937] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:34 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:34.432944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9c325-8cfe-45b8-936a-7e940bc26644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.434298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db6e5-e15a-495c-ad64-d28fb8e71d00 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.434418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9c325-8cfe-45b8-936a-7e940bc26644 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.444337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 967/1938] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:34.501177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db6e5-e15a-495c-ad64-d28fb8e71d00 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.505863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dd6f7-22ba-42fa-9dad-2ee6308f9c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.506479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dd6f7-22ba-42fa-9dad-2ee6308f9c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.506899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dd6f7-22ba-42fa-9dad-2ee6308f9c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.507212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dd6f7-22ba-42fa-9dad-2ee6308f9c30 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.507967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dd6f7-22ba-42fa-9dad-2ee6308f9c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.510504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dd6f7-22ba-42fa-9dad-2ee6308f9c30 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.538822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 972/1939] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:34 2022] POST /identity/v3/users => generated 301 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:34.543280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9c325-8cfe-45b8-936a-7e940bc26644 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.558350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9c325-8cfe-45b8-936a-7e940bc26644 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.568070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce81f56-438d-4cab-b664-5bed4c5ac620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.568274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce81f56-438d-4cab-b664-5bed4c5ac620 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.568556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce81f56-438d-4cab-b664-5bed4c5ac620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.568867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce81f56-438d-4cab-b664-5bed4c5ac620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.569561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce81f56-438d-4cab-b664-5bed4c5ac620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.573862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce81f56-438d-4cab-b664-5bed4c5ac620 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.590206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 968/1940] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:34.591473 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dd6f7-22ba-42fa-9dad-2ee6308f9c30 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.608890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dd6f7-22ba-42fa-9dad-2ee6308f9c30 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.631846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.632749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.633510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.633880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] PATH_INFO: `/v3/projects/046cbe91f5124f1a8d028d341d5654df/users/b1a2859edb37495799a684cd1d6c2591/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.654358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=046cbe91f5124f1a8d028d341d5654df, user_id=b1a2859edb37495799a684cd1d6c2591, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.657764 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.689417 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 973/1941] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:34 2022] PUT /identity/v3/projects/046cbe91f5124f1a8d028d341d5654df/users/b1a2859edb37495799a684cd1d6c2591/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:34.693366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce81f56-438d-4cab-b664-5bed4c5ac620 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.710023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce81f56-438d-4cab-b664-5bed4c5ac620 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.729094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:34.796643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.798608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.841911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:34.868512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8193106-e1f5-48f3-879a-c67aef13ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.870616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8193106-e1f5-48f3-879a-c67aef13ccc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.871518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8193106-e1f5-48f3-879a-c67aef13ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.871928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8193106-e1f5-48f3-879a-c67aef13ccc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.875540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8193106-e1f5-48f3-879a-c67aef13ccc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.877320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8193106-e1f5-48f3-879a-c67aef13ccc2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.889035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d970457-8063-4970-9880-a58628fb75ed admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.891256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 969/1942] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:34.894488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8193106-e1f5-48f3-879a-c67aef13ccc2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.898677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.903324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.905024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.905511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] PATH_INFO: `/v3/projects/0c0e1719bd29452795963e73e0ed888f` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.907653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0e1719bd29452795963e73e0ed888f)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.909189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8193106-e1f5-48f3-879a-c67aef13ccc2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.911501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.913675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d0d87-8a8d-4ee1-9297-e05f35dd15d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.914368 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d0d87-8a8d-4ee1-9297-e05f35dd15d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.914786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d0d87-8a8d-4ee1-9297-e05f35dd15d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.915094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d0d87-8a8d-4ee1-9297-e05f35dd15d1 admin admin] PATH_INFO: `/v3/projects/046cbe91f5124f1a8d028d341d5654df/users/b1a2859edb37495799a684cd1d6c2591/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.917276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d0d87-8a8d-4ee1-9297-e05f35dd15d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=046cbe91f5124f1a8d028d341d5654df, user_id=b1a2859edb37495799a684cd1d6c2591, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:34.919928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d0d87-8a8d-4ee1-9297-e05f35dd15d1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:34.934738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 970/1943] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:34 2022] PUT /identity/v3/projects/046cbe91f5124f1a8d028d341d5654df/users/b1a2859edb37495799a684cd1d6c2591/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:34.944846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebebd4d-06f5-43fb-9737-d6eb85ea4eee None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.945233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebebd4d-06f5-43fb-9737-d6eb85ea4eee None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.945508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebebd4d-06f5-43fb-9737-d6eb85ea4eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.983289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-eebebd4d-06f5-43fb-9737-d6eb85ea4eee None None] MFA Rules not processed for user `b1a2859edb37495799a684cd1d6c2591`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:34.984732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eebebd4d-06f5-43fb-9737-d6eb85ea4eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.992775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0e1719bd29452795963e73e0ed888f', 'request_id': 'req-a9fc8163-f6f5-466b-ad59-98fad03095a8'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.005600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0e1719bd29452795963e73e0ed888f', 'request_id': 'req-a9fc8163-f6f5-466b-ad59-98fad03095a8'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.015768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0e1719bd29452795963e73e0ed888f', 'request_id': 'req-a9fc8163-f6f5-466b-ad59-98fad03095a8'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.033673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] The token cache is being invalidate because project 0c0e1719bd29452795963e73e0ed888f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:35.034054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.034972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9fc8163-f6f5-466b-ad59-98fad03095a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.040506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 974/1944] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:34 2022] DELETE /identity/v3/projects/0c0e1719bd29452795963e73e0ed888f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:35.042852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3c113-16b8-43ca-a2de-dfab94bc27ee None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.044506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3c113-16b8-43ca-a2de-dfab94bc27ee None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.044506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3c113-16b8-43ca-a2de-dfab94bc27ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.079276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-58b3c113-16b8-43ca-a2de-dfab94bc27ee None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:35.083096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 971/1945] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:34 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:35.083223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-58b3c113-16b8-43ca-a2de-dfab94bc27ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.091680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebebd4d-06f5-43fb-9737-d6eb85ea4eee None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.094745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eebebd4d-06f5-43fb-9737-d6eb85ea4eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.171041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 975/1946] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:35.174222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a358f-a61a-4b48-ac80-f2f284a86a36 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.174530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a358f-a61a-4b48-ac80-f2f284a86a36 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.175035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a358f-a61a-4b48-ac80-f2f284a86a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.180197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebebd4d-06f5-43fb-9737-d6eb85ea4eee None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.184754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f2782-50cf-46c1-b0a0-3fed425b4920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.185373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f2782-50cf-46c1-b0a0-3fed425b4920 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.185661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f2782-50cf-46c1-b0a0-3fed425b4920 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.185935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f2782-50cf-46c1-b0a0-3fed425b4920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.195764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f2782-50cf-46c1-b0a0-3fed425b4920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.197372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f2782-50cf-46c1-b0a0-3fed425b4920 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.200165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ec1a358f-a61a-4b48-ac80-f2f284a86a36 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:35.201936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ec1a358f-a61a-4b48-ac80-f2f284a86a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.208648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 972/1947] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:35.211527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 976/1948] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:35.216037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f2782-50cf-46c1-b0a0-3fed425b4920 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.229608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f2782-50cf-46c1-b0a0-3fed425b4920 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.234249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.234850 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.235216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.235489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.236138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:35.236436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.238002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.255777 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 973/1949] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:35.256979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a358f-a61a-4b48-ac80-f2f284a86a36 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.268051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a358f-a61a-4b48-ac80-f2f284a86a36 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.274033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.274033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.274033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.274033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.274989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.277202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.316922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 977/1950] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:35 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:35.318658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.338604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc167be-165f-4063-96e9-61600af34076 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.349795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:35.361171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.363485 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.416678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:35.427697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868da38-3d23-42fe-b57a-afafc8d389c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.429233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868da38-3d23-42fe-b57a-afafc8d389c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.429589 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868da38-3d23-42fe-b57a-afafc8d389c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.429936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868da38-3d23-42fe-b57a-afafc8d389c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.430684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868da38-3d23-42fe-b57a-afafc8d389c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.433726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868da38-3d23-42fe-b57a-afafc8d389c8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.461979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 974/1951] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:35 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:35.464061 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868da38-3d23-42fe-b57a-afafc8d389c8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.465860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fc42f-a308-42f4-a1ed-c161f87748dd admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.472614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.474497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.474844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.475242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] PATH_INFO: `/v3/users/3dfafdac717849c2a8dae38c76eca624` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.476390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868da38-3d23-42fe-b57a-afafc8d389c8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.476650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfafdac717849c2a8dae38c76eca624)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.479506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.481882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60874cb2-c7cb-44c2-a09f-9fc7506baf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.482689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60874cb2-c7cb-44c2-a09f-9fc7506baf7a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.483014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60874cb2-c7cb-44c2-a09f-9fc7506baf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.483294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60874cb2-c7cb-44c2-a09f-9fc7506baf7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.483954 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60874cb2-c7cb-44c2-a09f-9fc7506baf7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.485781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60874cb2-c7cb-44c2-a09f-9fc7506baf7a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.501065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 975/1952] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:35.505057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60874cb2-c7cb-44c2-a09f-9fc7506baf7a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.517826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60874cb2-c7cb-44c2-a09f-9fc7506baf7a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.523882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa98a85-5e85-48d6-9bd9-9e446f3971e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.524557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa98a85-5e85-48d6-9bd9-9e446f3971e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.524931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa98a85-5e85-48d6-9bd9-9e446f3971e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.525735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa98a85-5e85-48d6-9bd9-9e446f3971e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.526349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa98a85-5e85-48d6-9bd9-9e446f3971e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.528154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa98a85-5e85-48d6-9bd9-9e446f3971e5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.539510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfafdac717849c2a8dae38c76eca624'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.540386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 976/1953] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:35.545083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa98a85-5e85-48d6-9bd9-9e446f3971e5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.551878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfafdac717849c2a8dae38c76eca624'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.553068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfafdac717849c2a8dae38c76eca624'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.554130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfafdac717849c2a8dae38c76eca624'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.555267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa98a85-5e85-48d6-9bd9-9e446f3971e5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.561601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b82f8-b46a-4c3f-9938-dc6d38530f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.562323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b82f8-b46a-4c3f-9938-dc6d38530f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.562638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b82f8-b46a-4c3f-9938-dc6d38530f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.563070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b82f8-b46a-4c3f-9938-dc6d38530f85 admin admin] PATH_INFO: `/v3/projects/a275a0cb529645fb9d5121e5818a0863/users/ffdbfc499e584dffb591aa1d6f64c4eb/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.565913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-15642188-b8aa-4528-982a-fc005ea85f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfafdac717849c2a8dae38c76eca624'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:35.579482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b82f8-b46a-4c3f-9938-dc6d38530f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a275a0cb529645fb9d5121e5818a0863, user_id=ffdbfc499e584dffb591aa1d6f64c4eb, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.583057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b82f8-b46a-4c3f-9938-dc6d38530f85 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.586232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 978/1954] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:35 2022] DELETE /identity/v3/users/3dfafdac717849c2a8dae38c76eca624 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:35.590762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15d32-5356-4517-a5f9-2b94658a5246 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.591071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15d32-5356-4517-a5f9-2b94658a5246 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.591429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15d32-5356-4517-a5f9-2b94658a5246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.602288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 977/1955] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:35 2022] PUT /identity/v3/projects/a275a0cb529645fb9d5121e5818a0863/users/ffdbfc499e584dffb591aa1d6f64c4eb/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:35.612682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-11c15d32-5356-4517-a5f9-2b94658a5246 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:35.613848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88617ed-afdd-4b94-98ba-330f91a6b284 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.614077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88617ed-afdd-4b94-98ba-330f91a6b284 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.614192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-11c15d32-5356-4517-a5f9-2b94658a5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.614529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88617ed-afdd-4b94-98ba-330f91a6b284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.646269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e88617ed-afdd-4b94-98ba-330f91a6b284 None None] MFA Rules not processed for user `ffdbfc499e584dffb591aa1d6f64c4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:35.647576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e88617ed-afdd-4b94-98ba-330f91a6b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.683204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 979/1956] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:35.704381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c15d32-5356-4517-a5f9-2b94658a5246 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.706011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-11c15d32-5356-4517-a5f9-2b94658a5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.725622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 978/1957] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:35.733926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88617ed-afdd-4b94-98ba-330f91a6b284 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.735512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e88617ed-afdd-4b94-98ba-330f91a6b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.778796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c15d32-5356-4517-a5f9-2b94658a5246 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.783677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.784481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.785012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.785426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.793252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:35.799656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.801106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:35.802464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.810741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88617ed-afdd-4b94-98ba-330f91a6b284 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.814822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3868d5-299e-4e56-80ed-41ccb1829bf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.815631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3868d5-299e-4e56-80ed-41ccb1829bf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.815914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3868d5-299e-4e56-80ed-41ccb1829bf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.816211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3868d5-299e-4e56-80ed-41ccb1829bf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.817094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 980/1958] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:35.825657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3868d5-299e-4e56-80ed-41ccb1829bf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.827144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3868d5-299e-4e56-80ed-41ccb1829bf1 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.836636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 979/1959] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:36.187848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:36.194061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:36.209086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865d73c-58fd-48f2-9fd8-f2f88a8d07f1 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:36.216161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:36.217090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:36.217548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:36.217923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] PATH_INFO: `/v3/users/8e63f18616d04c0a8fa5eb64582a80c4` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:36.219300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e63f18616d04c0a8fa5eb64582a80c4)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:36.221913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:36.282010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e63f18616d04c0a8fa5eb64582a80c4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.292426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e63f18616d04c0a8fa5eb64582a80c4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.293313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e63f18616d04c0a8fa5eb64582a80c4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.293921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e63f18616d04c0a8fa5eb64582a80c4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.305158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e63f18616d04c0a8fa5eb64582a80c4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.334515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 981/1960] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:36 2022] DELETE /identity/v3/users/8e63f18616d04c0a8fa5eb64582a80c4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:36.339181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a536f-f3d2-4696-a0a2-5000e330b88b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:36.339470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a536f-f3d2-4696-a0a2-5000e330b88b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:36.340325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a536f-f3d2-4696-a0a2-5000e330b88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:36.356371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:36.357942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:36.373510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b35a536f-f3d2-4696-a0a2-5000e330b88b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:36.375026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b35a536f-f3d2-4696-a0a2-5000e330b88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:36.473469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 980/1961] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:36.480061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf42f4-134a-4692-94b9-4bce60b7a678 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:36.485339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a536f-f3d2-4696-a0a2-5000e330b88b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:36.487140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b35a536f-f3d2-4696-a0a2-5000e330b88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:36.488877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:36.489922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:36.490349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:36.491069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] PATH_INFO: `/v3/projects/65f225fe4d0c46e7a00ccf16dd5fa88a` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:36.492495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f225fe4d0c46e7a00ccf16dd5fa88a)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:36.495233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:36.555927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f225fe4d0c46e7a00ccf16dd5fa88a', 'request_id': 'req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.562660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a536f-f3d2-4696-a0a2-5000e330b88b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:36.564551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f225fe4d0c46e7a00ccf16dd5fa88a', 'request_id': 'req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.572709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f225fe4d0c46e7a00ccf16dd5fa88a', 'request_id': 'req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.577420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:36.586220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] The token cache is being invalidate because project 65f225fe4d0c46e7a00ccf16dd5fa88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:36.586533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.587732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.589230 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 982/1962] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:36 2022] DELETE /identity/v3/projects/65f225fe4d0c46e7a00ccf16dd5fa88a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:36.592915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:36.595445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:36.626151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36febfdd-6749-4075-a000-02217815597c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:36.626660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36febfdd-6749-4075-a000-02217815597c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:36.626972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36febfdd-6749-4075-a000-02217815597c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:36.627366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36febfdd-6749-4075-a000-02217815597c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:36.629150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36febfdd-6749-4075-a000-02217815597c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:36.671324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0fe58-b2bc-481a-a3d6-8e27ef070254 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:36.676354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:36.676996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:36.677275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:36.677549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] PATH_INFO: `/v3/users/b1a2859edb37495799a684cd1d6c2591` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:36.679103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a2859edb37495799a684cd1d6c2591)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:36.681204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:36.700186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:36.705470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36febfdd-6749-4075-a000-02217815597c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:36.707410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36febfdd-6749-4075-a000-02217815597c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:36.720658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 981/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:36.734338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a2859edb37495799a684cd1d6c2591'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.741627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a2859edb37495799a684cd1d6c2591'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.742357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a2859edb37495799a684cd1d6c2591'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.743079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a2859edb37495799a684cd1d6c2591'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.755321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4695df21-0974-46ae-837f-9399c79e83e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a2859edb37495799a684cd1d6c2591'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:36.783715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 983/1964] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:36 2022] DELETE /identity/v3/users/b1a2859edb37495799a684cd1d6c2591 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:37.020515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36febfdd-6749-4075-a000-02217815597c service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:37.022424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36febfdd-6749-4075-a000-02217815597c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:37.043624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36febfdd-6749-4075-a000-02217815597c service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:37.167763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:37.168981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:37.170604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:37.171078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] PATH_INFO: `/v3/projects/fb07c878a90944a5874993d0d5d57b78` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:37.179761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb07c878a90944a5874993d0d5d57b78)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:37.181845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:37.299322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb07c878a90944a5874993d0d5d57b78', 'request_id': 'req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.318941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb07c878a90944a5874993d0d5d57b78', 'request_id': 'req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.330260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb07c878a90944a5874993d0d5d57b78', 'request_id': 'req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.368902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] The token cache is being invalidate because project fb07c878a90944a5874993d0d5d57b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:37.369399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.370937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.381567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 982/1965] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:37 2022] DELETE /identity/v3/projects/fb07c878a90944a5874993d0d5d57b78 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:37.383424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121eadb-7868-478d-a3da-696b54c7a2a1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:37.383924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121eadb-7868-478d-a3da-696b54c7a2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:37.384965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121eadb-7868-478d-a3da-696b54c7a2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:37.514751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9121eadb-7868-478d-a3da-696b54c7a2a1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:37.523355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9121eadb-7868-478d-a3da-696b54c7a2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:37.667490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:37.669239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:37.706028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 984/1966] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:37.714838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8137ef4-cef2-4b04-9116-2be99149ad4c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:37.717256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8137ef4-cef2-4b04-9116-2be99149ad4c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:37.717731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8137ef4-cef2-4b04-9116-2be99149ad4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:37.741099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a8137ef4-cef2-4b04-9116-2be99149ad4c None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:37.742379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a8137ef4-cef2-4b04-9116-2be99149ad4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:37.751796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 985/1967] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:37.755634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8137ef4-cef2-4b04-9116-2be99149ad4c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:37.766880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8137ef4-cef2-4b04-9116-2be99149ad4c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:37.772004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:37.772973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:37.773313 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:37.773679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:37.774779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:37.775680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:37.777334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:37.780869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:37.792034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bba2e-9ae2-4335-9ccf-678a1dab0402 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:37.793063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 986/1968] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:37.798543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:37.802365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:37.803064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:37.803342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:37.803876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] PATH_INFO: `/v3/projects/046cbe91f5124f1a8d028d341d5654df` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:37.805202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=046cbe91f5124f1a8d028d341d5654df)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:37.808340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:37.810924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a7be0-97a3-4903-a430-dc27cf4b4837 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:37.815160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f83b1-b546-4de3-824d-f9c8b16b2de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:37.815480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f83b1-b546-4de3-824d-f9c8b16b2de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:37.815758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f83b1-b546-4de3-824d-f9c8b16b2de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:37.816022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f83b1-b546-4de3-824d-f9c8b16b2de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:37.816641 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f83b1-b546-4de3-824d-f9c8b16b2de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:37.818686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f83b1-b546-4de3-824d-f9c8b16b2de8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:37.848235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:37.851349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 987/1969] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:37 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:37.855349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f83b1-b546-4de3-824d-f9c8b16b2de8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:37.880510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046cbe91f5124f1a8d028d341d5654df', 'request_id': 'req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.893287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046cbe91f5124f1a8d028d341d5654df', 'request_id': 'req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.907755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046cbe91f5124f1a8d028d341d5654df', 'request_id': 'req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.920075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] The token cache is being invalidate because project 046cbe91f5124f1a8d028d341d5654df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:37.920508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.921174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cf8be0f4-f6ca-4e25-b195-e44864f31c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:37.922692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 983/1970] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:37 2022] DELETE /identity/v3/projects/046cbe91f5124f1a8d028d341d5654df => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:37.928994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257b79b-1ded-4504-be80-8e47c02b8f79 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:37.929285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257b79b-1ded-4504-be80-8e47c02b8f79 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:37.929561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257b79b-1ded-4504-be80-8e47c02b8f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:37.948337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f83b1-b546-4de3-824d-f9c8b16b2de8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:37.955993 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4257b79b-1ded-4504-be80-8e47c02b8f79 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:37.957509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4257b79b-1ded-4504-be80-8e47c02b8f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.013483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.014097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.014407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.014674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.015332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.017271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.034933 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 984/1971] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:38.040796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 988/1972] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:37 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:38.044726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.045112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5fdca-61ea-436d-8fe9-1085030ccb75 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.045402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5fdca-61ea-436d-8fe9-1085030ccb75 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.045675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5fdca-61ea-436d-8fe9-1085030ccb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.046582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.059825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9939f52-d9bf-452d-b155-21e46d0f6d25 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.064783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725c569-1fd2-4c01-af05-fbb27eb7883f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.065386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c569-1fd2-4c01-af05-fbb27eb7883f admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.065664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c569-1fd2-4c01-af05-fbb27eb7883f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.065935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c569-1fd2-4c01-af05-fbb27eb7883f admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.066570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725c569-1fd2-4c01-af05-fbb27eb7883f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.068182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725c569-1fd2-4c01-af05-fbb27eb7883f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.068790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b7a5fdca-61ea-436d-8fe9-1085030ccb75 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:38.070266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b7a5fdca-61ea-436d-8fe9-1085030ccb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.079342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 989/1973] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:38.082075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 985/1974] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:38.083767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725c569-1fd2-4c01-af05-fbb27eb7883f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.086452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5fdca-61ea-436d-8fe9-1085030ccb75 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.094707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725c569-1fd2-4c01-af05-fbb27eb7883f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.095523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5fdca-61ea-436d-8fe9-1085030ccb75 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.100195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16757851-3345-49b9-85b4-bb5b9766c9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.100352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.100890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.101167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.101433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.101820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757851-3345-49b9-85b4-bb5b9766c9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.102072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:38.102370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.102762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757851-3345-49b9-85b4-bb5b9766c9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.103879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757851-3345-49b9-85b4-bb5b9766c9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.104515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.104947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16757851-3345-49b9-85b4-bb5b9766c9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.106719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16757851-3345-49b9-85b4-bb5b9766c9e0 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.117198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 986/1975] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:00:38.118310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 990/1976] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:38.121832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.123180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16757851-3345-49b9-85b4-bb5b9766c9e0 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.133650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16757851-3345-49b9-85b4-bb5b9766c9e0 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.134420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af54219-7cda-4e12-be57-417545c7d5af admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.139157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0725ba-323a-42ef-9695-d0529abd07a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.139407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bba20-5312-458d-a574-046c6dcf3b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.139756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0725ba-323a-42ef-9695-d0529abd07a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.140033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0725ba-323a-42ef-9695-d0529abd07a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.140302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0725ba-323a-42ef-9695-d0529abd07a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.140834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bba20-5312-458d-a574-046c6dcf3b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.140966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0725ba-323a-42ef-9695-d0529abd07a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.141424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bba20-5312-458d-a574-046c6dcf3b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.141894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bba20-5312-458d-a574-046c6dcf3b11 admin admin] PATH_INFO: `/v3/projects/5ed0620f14474ec4bd7db45569b71ccc/users/cc828fe19e2a41e791c8aad7d7b2548f/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.142810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0725ba-323a-42ef-9695-d0529abd07a7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.157881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bba20-5312-458d-a574-046c6dcf3b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed0620f14474ec4bd7db45569b71ccc, user_id=cc828fe19e2a41e791c8aad7d7b2548f, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.160624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bba20-5312-458d-a574-046c6dcf3b11 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.174082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 987/1977] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:38 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:38.181386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 991/1978] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:38 2022] PUT /identity/v3/projects/5ed0620f14474ec4bd7db45569b71ccc/users/cc828fe19e2a41e791c8aad7d7b2548f/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:38.183999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0725ba-323a-42ef-9695-d0529abd07a7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.186172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2330-111d-4b8e-ab8e-d5acae232bf2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.186631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2330-111d-4b8e-ab8e-d5acae232bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.187035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2330-111d-4b8e-ab8e-d5acae232bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.223849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c7df2330-111d-4b8e-ab8e-d5acae232bf2 None None] MFA Rules not processed for user `cc828fe19e2a41e791c8aad7d7b2548f`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:38.225169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c7df2330-111d-4b8e-ab8e-d5acae232bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.245041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:38.283282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0725ba-323a-42ef-9695-d0529abd07a7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.290087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.291379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.291981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.292370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.294485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.297376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.315623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 992/1979] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:38.328234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 988/1980] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:38 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:38.329120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.329231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df2330-111d-4b8e-ab8e-d5acae232bf2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.330917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.340033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:38.343116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df2330-111d-4b8e-ab8e-d5acae232bf2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.349065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6661b8a-f3d8-4896-9559-b3cf09a33735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.349695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6661b8a-f3d8-4896-9559-b3cf09a33735 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.349966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6661b8a-f3d8-4896-9559-b3cf09a33735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.350592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6661b8a-f3d8-4896-9559-b3cf09a33735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.351451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6661b8a-f3d8-4896-9559-b3cf09a33735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.353073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6661b8a-f3d8-4896-9559-b3cf09a33735 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.365911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 993/1981] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:38.373821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6661b8a-f3d8-4896-9559-b3cf09a33735 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.383249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6661b8a-f3d8-4896-9559-b3cf09a33735 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.388338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe885-fbc3-4c97-ae9c-f668d7eedede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.389021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe885-fbc3-4c97-ae9c-f668d7eedede admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.389222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe885-fbc3-4c97-ae9c-f668d7eedede admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.389496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe885-fbc3-4c97-ae9c-f668d7eedede admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.390172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbe885-fbc3-4c97-ae9c-f668d7eedede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.391980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbe885-fbc3-4c97-ae9c-f668d7eedede admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.405423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 994/1982] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:38.406649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b009797-0a14-485d-8a38-80949890e5c2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.409770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe885-fbc3-4c97-ae9c-f668d7eedede admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.411267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f807d43-4099-4f41-ba93-761f90360268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.411905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f807d43-4099-4f41-ba93-761f90360268 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.418178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f807d43-4099-4f41-ba93-761f90360268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.418510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f807d43-4099-4f41-ba93-761f90360268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.420671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe885-fbc3-4c97-ae9c-f668d7eedede admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.427040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63226b-079d-4aa2-a3df-cf35ba88c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.427781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63226b-079d-4aa2-a3df-cf35ba88c87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.428128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63226b-079d-4aa2-a3df-cf35ba88c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.428263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f807d43-4099-4f41-ba93-761f90360268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.428677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63226b-079d-4aa2-a3df-cf35ba88c87c admin admin] PATH_INFO: `/v3/projects/132383472a4f48779d8c4fc5a5179a8e/users/ef293aceb5e7483f9d69a4eab9e187fa/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.429845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f807d43-4099-4f41-ba93-761f90360268 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.441259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 989/1983] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:38.447977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63226b-079d-4aa2-a3df-cf35ba88c87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=132383472a4f48779d8c4fc5a5179a8e, user_id=ef293aceb5e7483f9d69a4eab9e187fa, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.450993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63226b-079d-4aa2-a3df-cf35ba88c87c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.470859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f807d43-4099-4f41-ba93-761f90360268 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.471260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 995/1984] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:38 2022] PUT /identity/v3/projects/132383472a4f48779d8c4fc5a5179a8e/users/ef293aceb5e7483f9d69a4eab9e187fa/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:38.482883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6de3-bdc0-4661-a4a4-13de1276ce50 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.483206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6de3-bdc0-4661-a4a4-13de1276ce50 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.483483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6de3-bdc0-4661-a4a4-13de1276ce50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.484533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f807d43-4099-4f41-ba93-761f90360268 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.518726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9cae6de3-bdc0-4661-a4a4-13de1276ce50 None None] MFA Rules not processed for user `ef293aceb5e7483f9d69a4eab9e187fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:38.520463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9cae6de3-bdc0-4661-a4a4-13de1276ce50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.608509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.609151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.609603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.609704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] PATH_INFO: `/v3/users/cc828fe19e2a41e791c8aad7d7b2548f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.615774 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc828fe19e2a41e791c8aad7d7b2548f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.618358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.626324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 996/1985] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:38.638503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae6de3-bdc0-4661-a4a4-13de1276ce50 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.702773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc828fe19e2a41e791c8aad7d7b2548f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:38.714347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc828fe19e2a41e791c8aad7d7b2548f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:38.717928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc828fe19e2a41e791c8aad7d7b2548f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:38.717928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc828fe19e2a41e791c8aad7d7b2548f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:38.734241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a9e6d5ba-2f9f-472a-ae0f-e2bc5eb3e67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc828fe19e2a41e791c8aad7d7b2548f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:38.757501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 990/1986] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:38 2022] DELETE /identity/v3/users/cc828fe19e2a41e791c8aad7d7b2548f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:38.763524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec0213-847c-4a76-b6f9-8b92b03d9699 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.763895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec0213-847c-4a76-b6f9-8b92b03d9699 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.764381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec0213-847c-4a76-b6f9-8b92b03d9699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.779691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae6de3-bdc0-4661-a4a4-13de1276ce50 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:38.792729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:38.803297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5bec0213-847c-4a76-b6f9-8b92b03d9699 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:38.806852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5bec0213-847c-4a76-b6f9-8b92b03d9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.837426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:38.865415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e80ab-1b93-4e6a-a4cc-1b178bc141bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:38.866046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e80ab-1b93-4e6a-a4cc-1b178bc141bd service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:38.866327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e80ab-1b93-4e6a-a4cc-1b178bc141bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:38.866596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e80ab-1b93-4e6a-a4cc-1b178bc141bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:38.869505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c05e80ab-1b93-4e6a-a4cc-1b178bc141bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.908848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 991/1987] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:38.929615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec0213-847c-4a76-b6f9-8b92b03d9699 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:38.931515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5bec0213-847c-4a76-b6f9-8b92b03d9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:38.957139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e80ab-1b93-4e6a-a4cc-1b178bc141bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:38.958871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e80ab-1b93-4e6a-a4cc-1b178bc141bd service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:38.971819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 997/1988] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:39.042637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec0213-847c-4a76-b6f9-8b92b03d9699 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:39.048102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d175e-9738-4968-bded-44f69cf80f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:39.049147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d175e-9738-4968-bded-44f69cf80f88 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.049147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d175e-9738-4968-bded-44f69cf80f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.049375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d175e-9738-4968-bded-44f69cf80f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.063297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d175e-9738-4968-bded-44f69cf80f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:39.064929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d175e-9738-4968-bded-44f69cf80f88 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:39.096111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 992/1989] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:39.378317 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae21f2-3da9-45d9-b17d-7f2660b74979 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.378606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae21f2-3da9-45d9-b17d-7f2660b74979 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.378948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae21f2-3da9-45d9-b17d-7f2660b74979 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.380797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 998/1990] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:39.393989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a18f13-0b5f-479a-9f3a-ae17611d8fc2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.394283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a18f13-0b5f-479a-9f3a-ae17611d8fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.394572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a18f13-0b5f-479a-9f3a-ae17611d8fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.455484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-97a18f13-0b5f-479a-9f3a-ae17611d8fc2 None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:39.456799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-97a18f13-0b5f-479a-9f3a-ae17611d8fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.478952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae21f2-3da9-45d9-b17d-7f2660b74979 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:39.480726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f8ae21f2-3da9-45d9-b17d-7f2660b74979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.505376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae21f2-3da9-45d9-b17d-7f2660b74979 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:39.514000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:39.514706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.515020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.515505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] PATH_INFO: `/v3/users/8568f2501f824875bae7ab1825cc2c32` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.516718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8568f2501f824875bae7ab1825cc2c32)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:39.518861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:39.618371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8568f2501f824875bae7ab1825cc2c32'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.620511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 993/1991] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 12:00:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:39.621237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a18f13-0b5f-479a-9f3a-ae17611d8fc2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:39.622635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-97a18f13-0b5f-479a-9f3a-ae17611d8fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.632304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8568f2501f824875bae7ab1825cc2c32'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.639332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a18f13-0b5f-479a-9f3a-ae17611d8fc2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:39.639548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8568f2501f824875bae7ab1825cc2c32'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.640992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8568f2501f824875bae7ab1825cc2c32'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.643855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:39.644523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.644805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.645073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] PATH_INFO: `/v3/projects/5ed0620f14474ec4bd7db45569b71ccc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.646369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed0620f14474ec4bd7db45569b71ccc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:39.648443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:39.653316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6c6ee74e-b656-445e-a152-a15bd52b7b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8568f2501f824875bae7ab1825cc2c32'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.679331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 999/1992] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:39 2022] DELETE /identity/v3/users/8568f2501f824875bae7ab1825cc2c32 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:39.683600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694baf7-49d8-4f99-8141-544ee74d4eaf None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.687126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694baf7-49d8-4f99-8141-544ee74d4eaf None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.687414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694baf7-49d8-4f99-8141-544ee74d4eaf None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.689326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1000/1993] 10.222.0.52 () {54 vars in 961 bytes} [Mon Oct 10 12:00:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:39.694999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12877d6-a4d7-4dd7-8aaa-248e60d1784f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.695404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12877d6-a4d7-4dd7-8aaa-248e60d1784f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.695754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12877d6-a4d7-4dd7-8aaa-248e60d1784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.721553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed0620f14474ec4bd7db45569b71ccc', 'request_id': 'req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.729822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f12877d6-a4d7-4dd7-8aaa-248e60d1784f None None] MFA Rules not processed for user `1e6c2f60d80647888998040638a5b993`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:39.731724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f12877d6-a4d7-4dd7-8aaa-248e60d1784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.736801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed0620f14474ec4bd7db45569b71ccc', 'request_id': 'req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.747355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed0620f14474ec4bd7db45569b71ccc', 'request_id': 'req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.748909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:39.763953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] The token cache is being invalidate because project 5ed0620f14474ec4bd7db45569b71ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:39.764275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.765225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aed4e3ab-ead1-44d7-90ce-6ae58e8fecc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:39.770842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 994/1994] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:39 2022] DELETE /identity/v3/projects/5ed0620f14474ec4bd7db45569b71ccc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:39.772997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11c0a-81bc-4cc3-846a-680bc464e30c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.773291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11c0a-81bc-4cc3-846a-680bc464e30c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.773569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11c0a-81bc-4cc3-846a-680bc464e30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.779122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:39.796966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bee11c0a-81bc-4cc3-846a-680bc464e30c None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:39.799569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bee11c0a-81bc-4cc3-846a-680bc464e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.816105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1001/1995] 10.222.0.52 () {58 vars in 1058 bytes} [Mon Oct 10 12:00:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:39.826830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12877d6-a4d7-4dd7-8aaa-248e60d1784f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:39.844257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12877d6-a4d7-4dd7-8aaa-248e60d1784f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:39.851322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804029d0-5934-4e1d-9230-9d272bad227e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:39.852256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804029d0-5934-4e1d-9230-9d272bad227e service placement] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.852369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804029d0-5934-4e1d-9230-9d272bad227e service placement] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.852697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804029d0-5934-4e1d-9230-9d272bad227e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.854788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-804029d0-5934-4e1d-9230-9d272bad227e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.869083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804029d0-5934-4e1d-9230-9d272bad227e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:39.871183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804029d0-5934-4e1d-9230-9d272bad227e service placement] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:39.882707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1002/1996] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:39.896734 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 995/1997] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:39.905465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa5f87-900e-40cb-869b-e9398ef7ceb7 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.905789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa5f87-900e-40cb-869b-e9398ef7ceb7 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.906128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa5f87-900e-40cb-869b-e9398ef7ceb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.929664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e5aa5f87-900e-40cb-869b-e9398ef7ceb7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:39.929799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee11c0a-81bc-4cc3-846a-680bc464e30c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:39.930856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e5aa5f87-900e-40cb-869b-e9398ef7ceb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.931223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bee11c0a-81bc-4cc3-846a-680bc464e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.940135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1003/1998] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:39.944309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa5f87-900e-40cb-869b-e9398ef7ceb7 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:39.944714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee11c0a-81bc-4cc3-846a-680bc464e30c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:39.951168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:39.951827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.952105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.952646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] PATH_INFO: `/v3/projects/d57fdf43bdef4f0dabd51175a84e9093` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.953817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57fdf43bdef4f0dabd51175a84e9093)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:39.955774 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:39.956928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa5f87-900e-40cb-869b-e9398ef7ceb7 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:39.968201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:39.969064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.969485 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.969946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.970994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:39.971476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:39.973438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:39.992051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1004/1999] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:00:39.993077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.004587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79085ed7-1a5f-4b19-8451-71d5912dabed admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.020972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57fdf43bdef4f0dabd51175a84e9093', 'request_id': 'req-ac1017d0-393e-4de7-8c49-f6f339a6f931'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:40.038371 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57fdf43bdef4f0dabd51175a84e9093', 'request_id': 'req-ac1017d0-393e-4de7-8c49-f6f339a6f931'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:40.046099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57fdf43bdef4f0dabd51175a84e9093', 'request_id': 'req-ac1017d0-393e-4de7-8c49-f6f339a6f931'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:40.058620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] The token cache is being invalidate because project d57fdf43bdef4f0dabd51175a84e9093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:40.058931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:40.059584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ac1017d0-393e-4de7-8c49-f6f339a6f931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:40.061122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 996/2000] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:39 2022] DELETE /identity/v3/projects/d57fdf43bdef4f0dabd51175a84e9093 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:40.065750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135337f-2853-4a17-85d5-904ad81bdeab None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.065886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1256a66-4150-4e87-8e1a-bb48e10aac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.066011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135337f-2853-4a17-85d5-904ad81bdeab None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.066285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135337f-2853-4a17-85d5-904ad81bdeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.066671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256a66-4150-4e87-8e1a-bb48e10aac92 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.067059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256a66-4150-4e87-8e1a-bb48e10aac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.067398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256a66-4150-4e87-8e1a-bb48e10aac92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.068086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1256a66-4150-4e87-8e1a-bb48e10aac92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.070070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1256a66-4150-4e87-8e1a-bb48e10aac92 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.086396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2135337f-2853-4a17-85d5-904ad81bdeab None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:40.087770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2135337f-2853-4a17-85d5-904ad81bdeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.096279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 997/2001] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:40.098637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1005/2002] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:39 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:40.102871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135337f-2853-4a17-85d5-904ad81bdeab None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.103841 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e61c-155e-47f0-b069-dfd956a30c86 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.104133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e61c-155e-47f0-b069-dfd956a30c86 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.104257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2135337f-2853-4a17-85d5-904ad81bdeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.104403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e61c-155e-47f0-b069-dfd956a30c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.116217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135337f-2853-4a17-85d5-904ad81bdeab None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.124304 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ff23e61c-155e-47f0-b069-dfd956a30c86 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:40.125479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ff23e61c-155e-47f0-b069-dfd956a30c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.174714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ac98-e2ff-4895-91cd-68689e0910b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.175276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ac98-e2ff-4895-91cd-68689e0910b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.175552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ac98-e2ff-4895-91cd-68689e0910b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.175820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ac98-e2ff-4895-91cd-68689e0910b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.176568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523ac98-e2ff-4895-91cd-68689e0910b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.178910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523ac98-e2ff-4895-91cd-68689e0910b9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.187294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1006/2003] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:40.191165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:40.191478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23e61c-155e-47f0-b069-dfd956a30c86 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.201823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23e61c-155e-47f0-b069-dfd956a30c86 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.205270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 998/2004] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:40 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:40.208971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.209116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ac98-e2ff-4895-91cd-68689e0910b9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.209645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.209953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.210242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.210964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:40.211265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.212973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.220226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ac98-e2ff-4895-91cd-68689e0910b9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.224947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c604fd9-f5d0-4291-9d3a-3d1349bdbbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.225387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1007/2005] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:00:40.227150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c604fd9-f5d0-4291-9d3a-3d1349bdbbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.227448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c604fd9-f5d0-4291-9d3a-3d1349bdbbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.227722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c604fd9-f5d0-4291-9d3a-3d1349bdbbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.228397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c604fd9-f5d0-4291-9d3a-3d1349bdbbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.229716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.229962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c604fd9-f5d0-4291-9d3a-3d1349bdbbf8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.239284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68780bbe-3f4c-42de-a59d-6a340eff44a8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.241369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 999/2006] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:40.244640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62865c5-c493-48c3-8e91-55a90e5cb950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.245254 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62865c5-c493-48c3-8e91-55a90e5cb950 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.245528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62865c5-c493-48c3-8e91-55a90e5cb950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.245644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c604fd9-f5d0-4291-9d3a-3d1349bdbbf8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.245789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62865c5-c493-48c3-8e91-55a90e5cb950 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.246402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62865c5-c493-48c3-8e91-55a90e5cb950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.248305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62865c5-c493-48c3-8e91-55a90e5cb950 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.255628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c604fd9-f5d0-4291-9d3a-3d1349bdbbf8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.260843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e01dbe-460d-4a42-99ea-1ac8a5bdea85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.261501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e01dbe-460d-4a42-99ea-1ac8a5bdea85 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.261778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e01dbe-460d-4a42-99ea-1ac8a5bdea85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.262048 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e01dbe-460d-4a42-99ea-1ac8a5bdea85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.262672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e01dbe-460d-4a42-99ea-1ac8a5bdea85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.264268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e01dbe-460d-4a42-99ea-1ac8a5bdea85 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.264924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:40.278884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1000/2007] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:40.279278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1008/2008] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:40 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:40.280168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e01dbe-460d-4a42-99ea-1ac8a5bdea85 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.283227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62865c5-c493-48c3-8e91-55a90e5cb950 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.289766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e01dbe-460d-4a42-99ea-1ac8a5bdea85 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.344194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62865c5-c493-48c3-8e91-55a90e5cb950 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.348799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1a884-de0f-4bdf-a4fc-ee0dcc606845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.349402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1a884-de0f-4bdf-a4fc-ee0dcc606845 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.349678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1a884-de0f-4bdf-a4fc-ee0dcc606845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.349944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1a884-de0f-4bdf-a4fc-ee0dcc606845 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.350563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1a884-de0f-4bdf-a4fc-ee0dcc606845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.352709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1a884-de0f-4bdf-a4fc-ee0dcc606845 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.357171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec073f-bf73-4bf4-b024-34a7ec7db84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.357835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec073f-bf73-4bf4-b024-34a7ec7db84f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.358115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec073f-bf73-4bf4-b024-34a7ec7db84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.358388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec073f-bf73-4bf4-b024-34a7ec7db84f admin admin] PATH_INFO: `/v3/projects/3ff498f4125745f2aef50f7ab265fee2/users/e2f735563ded4010b9af2124394435ae/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.374063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec073f-bf73-4bf4-b024-34a7ec7db84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff498f4125745f2aef50f7ab265fee2, user_id=e2f735563ded4010b9af2124394435ae, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.376265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1009/2009] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:40 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:40.376769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec073f-bf73-4bf4-b024-34a7ec7db84f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.380183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1a884-de0f-4bdf-a4fc-ee0dcc606845 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.389733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1001/2010] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:40 2022] PUT /identity/v3/projects/3ff498f4125745f2aef50f7ab265fee2/users/e2f735563ded4010b9af2124394435ae/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:40.399401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1a884-de0f-4bdf-a4fc-ee0dcc606845 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.401146 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6b46-613e-4b41-95ba-f742bd6c15b7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.401491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6b46-613e-4b41-95ba-f742bd6c15b7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.401768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6b46-613e-4b41-95ba-f742bd6c15b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.434582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f17f6b46-613e-4b41-95ba-f742bd6c15b7 None None] MFA Rules not processed for user `e2f735563ded4010b9af2124394435ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:40.435756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f17f6b46-613e-4b41-95ba-f742bd6c15b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.448320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3515b3-d0ea-4c1a-b528-eec0ea3ee2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.448934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3515b3-d0ea-4c1a-b528-eec0ea3ee2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.449209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3515b3-d0ea-4c1a-b528-eec0ea3ee2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.449478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3515b3-d0ea-4c1a-b528-eec0ea3ee2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.450111 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3515b3-d0ea-4c1a-b528-eec0ea3ee2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.451722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3515b3-d0ea-4c1a-b528-eec0ea3ee2d4 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.461371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1010/2011] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:40.465552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3515b3-d0ea-4c1a-b528-eec0ea3ee2d4 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.474726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3515b3-d0ea-4c1a-b528-eec0ea3ee2d4 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.480436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c412bd-4b07-4e6d-ad7b-b88b19bb1a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.481110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c412bd-4b07-4e6d-ad7b-b88b19bb1a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.481465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c412bd-4b07-4e6d-ad7b-b88b19bb1a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.481798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c412bd-4b07-4e6d-ad7b-b88b19bb1a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.482510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c412bd-4b07-4e6d-ad7b-b88b19bb1a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.484640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c412bd-4b07-4e6d-ad7b-b88b19bb1a02 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.494959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1011/2012] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:40.498924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c412bd-4b07-4e6d-ad7b-b88b19bb1a02 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.503420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1002/2013] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 10 12:00:40.508986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c412bd-4b07-4e6d-ad7b-b88b19bb1a02 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.513655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e369d71-55db-4bff-a281-3df7133416cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.513808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6b46-613e-4b41-95ba-f742bd6c15b7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.514632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e369d71-55db-4bff-a281-3df7133416cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.514938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e369d71-55db-4bff-a281-3df7133416cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.515165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f17f6b46-613e-4b41-95ba-f742bd6c15b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.515296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e369d71-55db-4bff-a281-3df7133416cd admin admin] PATH_INFO: `/v3/projects/abeb54035bdd44b4b04279b72be0e1e0/users/2f1e749fab7542469c7e53ef7d2cf632/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.529812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e369d71-55db-4bff-a281-3df7133416cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=abeb54035bdd44b4b04279b72be0e1e0, user_id=2f1e749fab7542469c7e53ef7d2cf632, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.532460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e369d71-55db-4bff-a281-3df7133416cd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.546664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1012/2014] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:40 2022] PUT /identity/v3/projects/abeb54035bdd44b4b04279b72be0e1e0/users/2f1e749fab7542469c7e53ef7d2cf632/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:40.557596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637f938-d106-4c79-b3c1-486e99137289 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.557945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637f938-d106-4c79-b3c1-486e99137289 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.558277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637f938-d106-4c79-b3c1-486e99137289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.575501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:40.583438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6b46-613e-4b41-95ba-f742bd6c15b7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.591664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9637f938-d106-4c79-b3c1-486e99137289 None None] MFA Rules not processed for user `2f1e749fab7542469c7e53ef7d2cf632`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:40.592903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9637f938-d106-4c79-b3c1-486e99137289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.629219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:40.649323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375155c8-2005-47e7-b4f0-fb6f164fc47c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.649942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375155c8-2005-47e7-b4f0-fb6f164fc47c service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.650247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375155c8-2005-47e7-b4f0-fb6f164fc47c service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.650520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375155c8-2005-47e7-b4f0-fb6f164fc47c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.675973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1013/2015] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:40.689269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637f938-d106-4c79-b3c1-486e99137289 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:40.690612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9637f938-d106-4c79-b3c1-486e99137289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.713116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375155c8-2005-47e7-b4f0-fb6f164fc47c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.714590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375155c8-2005-47e7-b4f0-fb6f164fc47c service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.725574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1003/2016] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:40.768472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637f938-d106-4c79-b3c1-486e99137289 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.776110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb31b65-a8fc-444c-a72e-c0a98530efba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.776747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb31b65-a8fc-444c-a72e-c0a98530efba service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.777617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb31b65-a8fc-444c-a72e-c0a98530efba service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.777896 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb31b65-a8fc-444c-a72e-c0a98530efba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.785850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb31b65-a8fc-444c-a72e-c0a98530efba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.787514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb31b65-a8fc-444c-a72e-c0a98530efba service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.797804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1014/2017] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:41.028902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375155c8-2005-47e7-b4f0-fb6f164fc47c service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:41.117089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375155c8-2005-47e7-b4f0-fb6f164fc47c service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:41.122969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:41.123703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:41.123990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:41.124265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] PATH_INFO: `/v3/users/2f1e749fab7542469c7e53ef7d2cf632` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:41.125448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1e749fab7542469c7e53ef7d2cf632)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:41.128002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:41.199543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1e749fab7542469c7e53ef7d2cf632'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.207429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1e749fab7542469c7e53ef7d2cf632'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.208349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1e749fab7542469c7e53ef7d2cf632'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.208992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1e749fab7542469c7e53ef7d2cf632'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.212017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:41.217961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1e749fab7542469c7e53ef7d2cf632'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.236340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1004/2018] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:41 2022] DELETE /identity/v3/users/2f1e749fab7542469c7e53ef7d2cf632 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:41.240786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9281b-7c9b-4c6c-a6da-dcb339b2ffdb None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:41.241075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9281b-7c9b-4c6c-a6da-dcb339b2ffdb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:41.241347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9281b-7c9b-4c6c-a6da-dcb339b2ffdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:41.260606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-1cd9281b-7c9b-4c6c-a6da-dcb339b2ffdb None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:41.261840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1cd9281b-7c9b-4c6c-a6da-dcb339b2ffdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:41.321153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1015/2019] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:41.331371 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:41.332791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:41.391577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898905c-78b9-4b73-9024-99109267a818 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:41.395600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:41.396201 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:41.396482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:41.396754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:41.406092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:41.407557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:41.418393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1005/2020] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:41.670535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2427e2b-cbfa-42ea-b00b-dfbc95c4e5c4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:41.671087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2427e2b-cbfa-42ea-b00b-dfbc95c4e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:41.671087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2427e2b-cbfa-42ea-b00b-dfbc95c4e5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:41.682552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:41.684316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:41.697497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e034d4-d2e1-47b7-a75c-972f74fa4f19 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:41.702159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:41.702857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:41.703256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:41.703612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] PATH_INFO: `/v3/projects/abeb54035bdd44b4b04279b72be0e1e0` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:41.704783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=abeb54035bdd44b4b04279b72be0e1e0)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:41.707334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:41.709921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a2427e2b-cbfa-42ea-b00b-dfbc95c4e5c4 None None] MFA Rules not processed for user `ef293aceb5e7483f9d69a4eab9e187fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:41.711445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a2427e2b-cbfa-42ea-b00b-dfbc95c4e5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:41.764942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abeb54035bdd44b4b04279b72be0e1e0', 'request_id': 'req-26d547e9-9111-48eb-ac13-1d2431a300fa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.775923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abeb54035bdd44b4b04279b72be0e1e0', 'request_id': 'req-26d547e9-9111-48eb-ac13-1d2431a300fa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.785229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abeb54035bdd44b4b04279b72be0e1e0', 'request_id': 'req-26d547e9-9111-48eb-ac13-1d2431a300fa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.789144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1016/2021] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:41.797687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] The token cache is being invalidate because project abeb54035bdd44b4b04279b72be0e1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:41.797975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.798717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-26d547e9-9111-48eb-ac13-1d2431a300fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:41.800458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1006/2022] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:41 2022] DELETE /identity/v3/projects/abeb54035bdd44b4b04279b72be0e1e0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:41.807329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e678f-af0e-4659-a488-1f94231e33ef None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:41.807685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e678f-af0e-4659-a488-1f94231e33ef None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:41.808024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e678f-af0e-4659-a488-1f94231e33ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:41.817470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:41.830354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-178e678f-af0e-4659-a488-1f94231e33ef None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:41.831447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-178e678f-af0e-4659-a488-1f94231e33ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:41.918686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1017/2023] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:41.927878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596264-8289-4546-b5d0-3095a028d3f7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:41.928165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596264-8289-4546-b5d0-3095a028d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:41.928723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596264-8289-4546-b5d0-3095a028d3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:41.949803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-64596264-8289-4546-b5d0-3095a028d3f7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:41.951050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-64596264-8289-4546-b5d0-3095a028d3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:41.958938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1007/2024] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:41.963277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e678f-af0e-4659-a488-1f94231e33ef None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:41.973647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e678f-af0e-4659-a488-1f94231e33ef None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:41.978761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:41.979409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:41.979688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:41.979959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:41.980601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:41.980903 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:41.982492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:41.997895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1018/2025] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:00:41.999134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64596264-8289-4546-b5d0-3095a028d3f7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:42.008311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64596264-8289-4546-b5d0-3095a028d3f7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:42.013016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f225e52-a60c-4de8-af1a-556b1b077792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:42.013615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f225e52-a60c-4de8-af1a-556b1b077792 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:42.013893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f225e52-a60c-4de8-af1a-556b1b077792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:42.014158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f225e52-a60c-4de8-af1a-556b1b077792 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:42.014774 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f225e52-a60c-4de8-af1a-556b1b077792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:42.016689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f225e52-a60c-4de8-af1a-556b1b077792 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:42.048142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1008/2026] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:41 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:42.050185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:42.061105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5813ed2-6a21-4597-b047-2c29919e4072 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:42.118126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891c1b2-6911-4ac5-9441-4433b2c52b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:42.119342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c1b2-6911-4ac5-9441-4433b2c52b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:42.119643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c1b2-6911-4ac5-9441-4433b2c52b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:42.119913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c1b2-6911-4ac5-9441-4433b2c52b4c admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:42.120540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891c1b2-6911-4ac5-9441-4433b2c52b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:42.122456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891c1b2-6911-4ac5-9441-4433b2c52b4c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:42.143184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1019/2027] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:42 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:42.148830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f225e52-a60c-4de8-af1a-556b1b077792 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:42.163931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f225e52-a60c-4de8-af1a-556b1b077792 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:42.173317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebd35-04e6-480d-adb2-bb0d59f365ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:42.175044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebd35-04e6-480d-adb2-bb0d59f365ea admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:42.175403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebd35-04e6-480d-adb2-bb0d59f365ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:42.175674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebd35-04e6-480d-adb2-bb0d59f365ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:42.176333 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cebd35-04e6-480d-adb2-bb0d59f365ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:42.177967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cebd35-04e6-480d-adb2-bb0d59f365ea admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:42.189505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1009/2028] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:42.195719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891c1b2-6911-4ac5-9441-4433b2c52b4c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:42.211906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891c1b2-6911-4ac5-9441-4433b2c52b4c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:42.218170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89eb7f4-b1c6-41b5-839a-5ad881fa9186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:42.219025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eb7f4-b1c6-41b5-839a-5ad881fa9186 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:42.219381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eb7f4-b1c6-41b5-839a-5ad881fa9186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:42.220036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eb7f4-b1c6-41b5-839a-5ad881fa9186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:42.220434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89eb7f4-b1c6-41b5-839a-5ad881fa9186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:42.222117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89eb7f4-b1c6-41b5-839a-5ad881fa9186 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:42.246139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1020/2029] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:42.249739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebd35-04e6-480d-adb2-bb0d59f365ea admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:42.275591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebd35-04e6-480d-adb2-bb0d59f365ea admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:42.282119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:42.283609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:42.284135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:42.284693 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] PATH_INFO: `/v3/projects/45628e5deeb045e4bdf7bc3b53555d67/users/2c92e7453b1645b29c6c8ad7c38b7735/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:42.305279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] RBAC: Authorizing `identity:create_grant(project_id=45628e5deeb045e4bdf7bc3b53555d67, user_id=2c92e7453b1645b29c6c8ad7c38b7735, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:42.308641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:42.320683 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:42.335561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1010/2030] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:42 2022] PUT /identity/v3/projects/45628e5deeb045e4bdf7bc3b53555d67/users/2c92e7453b1645b29c6c8ad7c38b7735/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:42.337244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b560131-271a-4b08-9bb0-2c59b9fd0608 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:42.338092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b560131-271a-4b08-9bb0-2c59b9fd0608 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:42.338583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b560131-271a-4b08-9bb0-2c59b9fd0608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:42.364523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-0b560131-271a-4b08-9bb0-2c59b9fd0608 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:42.366276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0b560131-271a-4b08-9bb0-2c59b9fd0608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:42.464431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1021/2031] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:42.476093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:42.478137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:42.574032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f1c8f-929a-4d7a-8c28-d310222023be admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:42.578517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c4854-3284-4d8f-8a69-f89edd2a4409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:42.579355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c4854-3284-4d8f-8a69-f89edd2a4409 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:42.579860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c4854-3284-4d8f-8a69-f89edd2a4409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:42.580258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c4854-3284-4d8f-8a69-f89edd2a4409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:42.595620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c4854-3284-4d8f-8a69-f89edd2a4409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:42.600338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c4854-3284-4d8f-8a69-f89edd2a4409 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:42.622044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1011/2032] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:43.310103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632931d-d65b-4fdc-87f6-11913dc6ca9b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.310591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632931d-d65b-4fdc-87f6-11913dc6ca9b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.310682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632931d-d65b-4fdc-87f6-11913dc6ca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.332048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:43.340781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-0632931d-d65b-4fdc-87f6-11913dc6ca9b None None] MFA Rules not processed for user `2c92e7453b1645b29c6c8ad7c38b7735`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:43.342747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0632931d-d65b-4fdc-87f6-11913dc6ca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:43.415025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1022/2033] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:43.420149 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c4854-3284-4d8f-8a69-f89edd2a4409 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:43.430392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c4854-3284-4d8f-8a69-f89edd2a4409 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:43.435717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf590aa-1d35-4704-9831-7848f29e580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:43.436320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf590aa-1d35-4704-9831-7848f29e580c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.436601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf590aa-1d35-4704-9831-7848f29e580c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.436872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf590aa-1d35-4704-9831-7848f29e580c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.437508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf590aa-1d35-4704-9831-7848f29e580c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:43.439441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf590aa-1d35-4704-9831-7848f29e580c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:43.469168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1012/2034] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:43 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:43.471040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632931d-d65b-4fdc-87f6-11913dc6ca9b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:43.482147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632931d-d65b-4fdc-87f6-11913dc6ca9b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:43.537630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78564b32-32a7-4b53-a4b6-68a6f7d88ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:43.538252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78564b32-32a7-4b53-a4b6-68a6f7d88ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.538535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78564b32-32a7-4b53-a4b6-68a6f7d88ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.538808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78564b32-32a7-4b53-a4b6-68a6f7d88ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.539495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78564b32-32a7-4b53-a4b6-68a6f7d88ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:43.541944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78564b32-32a7-4b53-a4b6-68a6f7d88ad2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:43.563320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1023/2035] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:43 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:43.567340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf590aa-1d35-4704-9831-7848f29e580c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:43.577635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf590aa-1d35-4704-9831-7848f29e580c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:43.582265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026fe85-81a6-4c8b-b13d-eeb595b4fff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:43.582805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026fe85-81a6-4c8b-b13d-eeb595b4fff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.583118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026fe85-81a6-4c8b-b13d-eeb595b4fff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.583388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026fe85-81a6-4c8b-b13d-eeb595b4fff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.584023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026fe85-81a6-4c8b-b13d-eeb595b4fff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:43.585618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026fe85-81a6-4c8b-b13d-eeb595b4fff1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:43.596021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1013/2036] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:43.599971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78564b32-32a7-4b53-a4b6-68a6f7d88ad2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:43.610997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78564b32-32a7-4b53-a4b6-68a6f7d88ad2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:43.615695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796a382-4127-43e6-928b-ec09bd037302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:43.616348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796a382-4127-43e6-928b-ec09bd037302 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.616663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796a382-4127-43e6-928b-ec09bd037302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.616939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796a382-4127-43e6-928b-ec09bd037302 admin admin] PATH_INFO: `/v3/projects/50cb17ecff894cbe800d59cfef6042ac/users/9da6cc2bfc99435296c186a10f89b5f6/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.632090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796a382-4127-43e6-928b-ec09bd037302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cb17ecff894cbe800d59cfef6042ac, user_id=9da6cc2bfc99435296c186a10f89b5f6, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:43.634870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796a382-4127-43e6-928b-ec09bd037302 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:43.647524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1024/2037] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:43 2022] PUT /identity/v3/projects/50cb17ecff894cbe800d59cfef6042ac/users/9da6cc2bfc99435296c186a10f89b5f6/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:43.651658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026fe85-81a6-4c8b-b13d-eeb595b4fff1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:43.661664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026fe85-81a6-4c8b-b13d-eeb595b4fff1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:43.722964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c04659-b806-45fd-ba16-864bb1e3b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:43.723607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c04659-b806-45fd-ba16-864bb1e3b362 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.723889 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c04659-b806-45fd-ba16-864bb1e3b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.724161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c04659-b806-45fd-ba16-864bb1e3b362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.724810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c04659-b806-45fd-ba16-864bb1e3b362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:43.726526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c04659-b806-45fd-ba16-864bb1e3b362 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:43.737591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1014/2038] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:43.741940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796a382-4127-43e6-928b-ec09bd037302 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:43.752473 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796a382-4127-43e6-928b-ec09bd037302 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:43.757190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:43.757842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.758122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.758391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] PATH_INFO: `/v3/projects/50cb17ecff894cbe800d59cfef6042ac/users/9da6cc2bfc99435296c186a10f89b5f6/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.760305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cb17ecff894cbe800d59cfef6042ac, user_id=9da6cc2bfc99435296c186a10f89b5f6, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:43.763017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:43.775430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1025/2039] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:43 2022] PUT /identity/v3/projects/50cb17ecff894cbe800d59cfef6042ac/users/9da6cc2bfc99435296c186a10f89b5f6/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:44.403165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226f9c4-4df4-4bca-ba9a-c91ff6f40533 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:44.403459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226f9c4-4df4-4bca-ba9a-c91ff6f40533 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:44.403736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226f9c4-4df4-4bca-ba9a-c91ff6f40533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:44.436795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c226f9c4-4df4-4bca-ba9a-c91ff6f40533 None None] MFA Rules not processed for user `9da6cc2bfc99435296c186a10f89b5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:44.437916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c226f9c4-4df4-4bca-ba9a-c91ff6f40533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:44.488192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:44.509913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1015/2040] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:44.521055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:44.522755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:44.580658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123e529-438b-4b6b-a275-52e4d784596a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:44.584826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:44.585537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:44.585915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:44.586301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:44.592368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:44.643204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:44.644640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:44.653774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1026/2041] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:46.078387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226f9c4-4df4-4bca-ba9a-c91ff6f40533 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:46.089251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226f9c4-4df4-4bca-ba9a-c91ff6f40533 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:46.094010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fddf24-6e39-4969-8e52-5f8af15875ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:46.094758 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fddf24-6e39-4969-8e52-5f8af15875ed service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:46.095104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fddf24-6e39-4969-8e52-5f8af15875ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:46.095388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fddf24-6e39-4969-8e52-5f8af15875ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:46.104961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fddf24-6e39-4969-8e52-5f8af15875ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:46.106498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fddf24-6e39-4969-8e52-5f8af15875ed service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:46.116302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1016/2042] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:53.173801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:53.175278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:53.240580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb8e16-ece6-4b72-91cf-d5f56512109d service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:53.245765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:53.246463 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:53.246738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:53.247033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] PATH_INFO: `/v3/users/e2f735563ded4010b9af2124394435ae` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:53.248135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f735563ded4010b9af2124394435ae)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:53.250040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:53.294276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f735563ded4010b9af2124394435ae'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.301236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f735563ded4010b9af2124394435ae'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.301915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f735563ded4010b9af2124394435ae'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.302526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f735563ded4010b9af2124394435ae'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.311356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f735563ded4010b9af2124394435ae'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.328128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1027/2043] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:53 2022] DELETE /identity/v3/users/e2f735563ded4010b9af2124394435ae => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:53.332679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea051-dc70-41d6-b71c-8a77607cf314 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:53.333218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea051-dc70-41d6-b71c-8a77607cf314 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:53.333218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea051-dc70-41d6-b71c-8a77607cf314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:53.355123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fa8ea051-dc70-41d6-b71c-8a77607cf314 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:53.355123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fa8ea051-dc70-41d6-b71c-8a77607cf314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:53.414495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1017/2044] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:53.424275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:53.426105 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:53.456909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:53.490512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb295b1-f1a4-4a8b-aaa3-ae145efd267b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:53.508152 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651d1ed-0197-4e10-94f7-61c47fe85d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:53.508792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651d1ed-0197-4e10-94f7-61c47fe85d9f service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:53.509107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651d1ed-0197-4e10-94f7-61c47fe85d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:53.509379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651d1ed-0197-4e10-94f7-61c47fe85d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:53.520584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651d1ed-0197-4e10-94f7-61c47fe85d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:53.522021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651d1ed-0197-4e10-94f7-61c47fe85d9f service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:53.544879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1028/2045] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:53.887517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ea051-dc70-41d6-b71c-8a77607cf314 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:53.889361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fa8ea051-dc70-41d6-b71c-8a77607cf314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:53.902919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ea051-dc70-41d6-b71c-8a77607cf314 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:53.908033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:53.908846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:53.909118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:53.909386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] PATH_INFO: `/v3/projects/3ff498f4125745f2aef50f7ab265fee2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:53.910577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff498f4125745f2aef50f7ab265fee2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:53.912539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:53.963202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff498f4125745f2aef50f7ab265fee2', 'request_id': 'req-4c59f852-55ab-4db1-b403-08624c1fff46'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.970677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff498f4125745f2aef50f7ab265fee2', 'request_id': 'req-4c59f852-55ab-4db1-b403-08624c1fff46'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.977190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff498f4125745f2aef50f7ab265fee2', 'request_id': 'req-4c59f852-55ab-4db1-b403-08624c1fff46'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.987501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] The token cache is being invalidate because project 3ff498f4125745f2aef50f7ab265fee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:53.987785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.988425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4c59f852-55ab-4db1-b403-08624c1fff46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:53.993394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1018/2046] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:53 2022] DELETE /identity/v3/projects/3ff498f4125745f2aef50f7ab265fee2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:53.995165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310805-71fc-4c23-b7f6-933984eb0002 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:53.995526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310805-71fc-4c23-b7f6-933984eb0002 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:53.995800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310805-71fc-4c23-b7f6-933984eb0002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.015493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-2d310805-71fc-4c23-b7f6-933984eb0002 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:54.016691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2d310805-71fc-4c23-b7f6-933984eb0002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:54.078857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1029/2047] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:54.086944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1358fe-4580-4654-acd0-7ddd8ac3cf70 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.087795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1358fe-4580-4654-acd0-7ddd8ac3cf70 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.088080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1358fe-4580-4654-acd0-7ddd8ac3cf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.109160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5b1358fe-4580-4654-acd0-7ddd8ac3cf70 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:54.110376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b1358fe-4580-4654-acd0-7ddd8ac3cf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:54.120060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1019/2048] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:54.124436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d310805-71fc-4c23-b7f6-933984eb0002 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:54.134534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d310805-71fc-4c23-b7f6-933984eb0002 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:54.139101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:54.139699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.139974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.140242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.140873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:54.141169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:54.142744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:54.154811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1030/2049] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:54.159209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1358fe-4580-4654-acd0-7ddd8ac3cf70 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:54.172055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1358fe-4580-4654-acd0-7ddd8ac3cf70 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:54.177119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3b15-a3fa-483a-b340-cacf6287b860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:54.177719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3b15-a3fa-483a-b340-cacf6287b860 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.178005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3b15-a3fa-483a-b340-cacf6287b860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.178543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a3b15-a3fa-483a-b340-cacf6287b860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.179215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a3b15-a3fa-483a-b340-cacf6287b860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:54.181623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a3b15-a3fa-483a-b340-cacf6287b860 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:54.208632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1020/2050] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:54 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:54.212775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:54.222684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc23930-39af-4f34-898e-626674dee123 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:54.283015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb63fd-87e0-45f4-8c88-a00356deb565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:54.283698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb63fd-87e0-45f4-8c88-a00356deb565 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.283974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb63fd-87e0-45f4-8c88-a00356deb565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.284242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb63fd-87e0-45f4-8c88-a00356deb565 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.284861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb63fd-87e0-45f4-8c88-a00356deb565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:54.286779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb63fd-87e0-45f4-8c88-a00356deb565 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:54.309753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1031/2051] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:54 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:54.313790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3b15-a3fa-483a-b340-cacf6287b860 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:54.324629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a3b15-a3fa-483a-b340-cacf6287b860 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:54.329320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932924-a9d2-49b8-81c9-7a3e8d514f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:54.329919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932924-a9d2-49b8-81c9-7a3e8d514f87 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.330195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932924-a9d2-49b8-81c9-7a3e8d514f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.330462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932924-a9d2-49b8-81c9-7a3e8d514f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.331129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932924-a9d2-49b8-81c9-7a3e8d514f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:54.332691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932924-a9d2-49b8-81c9-7a3e8d514f87 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:54.342731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1021/2052] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:54.346697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb63fd-87e0-45f4-8c88-a00356deb565 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:54.356433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb63fd-87e0-45f4-8c88-a00356deb565 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:54.361139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca246d2-412b-4a6e-8b3c-d9816c926298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:54.361730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca246d2-412b-4a6e-8b3c-d9816c926298 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.362003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca246d2-412b-4a6e-8b3c-d9816c926298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.362270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca246d2-412b-4a6e-8b3c-d9816c926298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.362922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca246d2-412b-4a6e-8b3c-d9816c926298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:54.364515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca246d2-412b-4a6e-8b3c-d9816c926298 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:54.375714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1032/2053] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:54.379819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932924-a9d2-49b8-81c9-7a3e8d514f87 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:54.390423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932924-a9d2-49b8-81c9-7a3e8d514f87 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:54.395145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:54.395828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.396175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.396521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] PATH_INFO: `/v3/projects/c3c9f5154d5e4f61aa9f2d731f1c772d/users/f286272b22da470f942aba726058c626/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.424805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9f5154d5e4f61aa9f2d731f1c772d, user_id=f286272b22da470f942aba726058c626, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:54.427717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:54.431822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:54.444269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1022/2054] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:54 2022] PUT /identity/v3/projects/c3c9f5154d5e4f61aa9f2d731f1c772d/users/f286272b22da470f942aba726058c626/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:00:54.456197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ba663-d8a8-4e6f-94ac-06a47fceabb4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.456484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ba663-d8a8-4e6f-94ac-06a47fceabb4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.456774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ba663-d8a8-4e6f-94ac-06a47fceabb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.483774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-db5ba663-d8a8-4e6f-94ac-06a47fceabb4 None None] MFA Rules not processed for user `f286272b22da470f942aba726058c626`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:54.484983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-db5ba663-d8a8-4e6f-94ac-06a47fceabb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:54.507997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:54.549435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1033/2055] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:54.561746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:54.563555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:54.623855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbac9d-77c6-4c2c-a201-aa6e38158c79 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:54.627522 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002213b9-150e-4e4a-8eda-a79e2317ffb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:54.628135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002213b9-150e-4e4a-8eda-a79e2317ffb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:54.628411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002213b9-150e-4e4a-8eda-a79e2317ffb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:54.628679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002213b9-150e-4e4a-8eda-a79e2317ffb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:54.637162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002213b9-150e-4e4a-8eda-a79e2317ffb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:54.638689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002213b9-150e-4e4a-8eda-a79e2317ffb4 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:54.649392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1023/2056] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:54 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:55.852661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ba663-d8a8-4e6f-94ac-06a47fceabb4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:55.854073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-db5ba663-d8a8-4e6f-94ac-06a47fceabb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:55.920205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ba663-d8a8-4e6f-94ac-06a47fceabb4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:55.924640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:55.925344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:55.925639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:55.925896 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] PATH_INFO: `/v3/users/ffdbfc499e584dffb591aa1d6f64c4eb` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:55.927309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffdbfc499e584dffb591aa1d6f64c4eb)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:55.929558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:55.986988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffdbfc499e584dffb591aa1d6f64c4eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.005097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffdbfc499e584dffb591aa1d6f64c4eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.006098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffdbfc499e584dffb591aa1d6f64c4eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.007256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffdbfc499e584dffb591aa1d6f64c4eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.018971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffdbfc499e584dffb591aa1d6f64c4eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.047608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1034/2057] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:00:55 2022] DELETE /identity/v3/users/ffdbfc499e584dffb591aa1d6f64c4eb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:56.049645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641d864-4d6e-4f77-a520-a7786f56a41f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:56.050101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641d864-4d6e-4f77-a520-a7786f56a41f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:56.050196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641d864-4d6e-4f77-a520-a7786f56a41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:56.070269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b641d864-4d6e-4f77-a520-a7786f56a41f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:56.072051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b641d864-4d6e-4f77-a520-a7786f56a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:56.179218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1024/2058] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:56.193525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:56.201911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:56.260157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:56.278744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebccf9-400e-4ea5-97b1-190eda2f7260 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:56.284739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5756a-595f-4fa7-b56f-6512b08bdf38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:56.285487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5756a-595f-4fa7-b56f-6512b08bdf38 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:56.285773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5756a-595f-4fa7-b56f-6512b08bdf38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:56.286047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5756a-595f-4fa7-b56f-6512b08bdf38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:56.307697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5756a-595f-4fa7-b56f-6512b08bdf38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:56.310805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5756a-595f-4fa7-b56f-6512b08bdf38 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:56.322409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1035/2059] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:56.656363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641d864-4d6e-4f77-a520-a7786f56a41f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:56.657895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b641d864-4d6e-4f77-a520-a7786f56a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:56.672382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641d864-4d6e-4f77-a520-a7786f56a41f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:56.677292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:56.679316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:56.679603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:56.679961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] PATH_INFO: `/v3/projects/a275a0cb529645fb9d5121e5818a0863` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:56.683309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=a275a0cb529645fb9d5121e5818a0863)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:56.683309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:56.736702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a275a0cb529645fb9d5121e5818a0863', 'request_id': 'req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.744471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a275a0cb529645fb9d5121e5818a0863', 'request_id': 'req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.751381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a275a0cb529645fb9d5121e5818a0863', 'request_id': 'req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.762110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] The token cache is being invalidate because project a275a0cb529645fb9d5121e5818a0863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:00:56.762393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.763397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1768c4e9-f7e8-4ab9-a5ab-c9ac7868b0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:00:56.765090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1025/2060] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:00:56 2022] DELETE /identity/v3/projects/a275a0cb529645fb9d5121e5818a0863 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:56.770079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeecb-9fd4-4384-97dd-10f0aab4a2f1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:56.770347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeecb-9fd4-4384-97dd-10f0aab4a2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:56.770611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeecb-9fd4-4384-97dd-10f0aab4a2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:56.790832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-900eeecb-9fd4-4384-97dd-10f0aab4a2f1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:56.792068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-900eeecb-9fd4-4384-97dd-10f0aab4a2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:56.850677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1036/2061] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:56.858436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca22bd1-c22b-424f-afa5-9897112b60b5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:56.858752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca22bd1-c22b-424f-afa5-9897112b60b5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:56.859179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca22bd1-c22b-424f-afa5-9897112b60b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:56.879417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1ca22bd1-c22b-424f-afa5-9897112b60b5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:56.880601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1ca22bd1-c22b-424f-afa5-9897112b60b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:56.888959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1026/2062] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:56.893019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900eeecb-9fd4-4384-97dd-10f0aab4a2f1 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:56.902983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900eeecb-9fd4-4384-97dd-10f0aab4a2f1 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:56.907553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:56.908141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:56.908414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:56.908680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:56.909312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:00:56.909607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:56.911216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:56.922461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1037/2063] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:00:56.926538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca22bd1-c22b-424f-afa5-9897112b60b5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:56.936122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca22bd1-c22b-424f-afa5-9897112b60b5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:56.940775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a3fb7-f309-468c-af1a-ca31dc959087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:56.941364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a3fb7-f309-468c-af1a-ca31dc959087 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:56.941642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a3fb7-f309-468c-af1a-ca31dc959087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:56.941914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a3fb7-f309-468c-af1a-ca31dc959087 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:56.942534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a3fb7-f309-468c-af1a-ca31dc959087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:56.944470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a3fb7-f309-468c-af1a-ca31dc959087 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:56.959114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:56.973722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1027/2064] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:00:56 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:00:56.979606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:56.989665 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7547a3-32e0-4d8c-8d5d-ed9699901575 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:57.043993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dad63a-09e0-4c4b-aaaa-659afa573e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:57.044516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dad63a-09e0-4c4b-aaaa-659afa573e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:57.044790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dad63a-09e0-4c4b-aaaa-659afa573e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:57.045054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dad63a-09e0-4c4b-aaaa-659afa573e30 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:57.045670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dad63a-09e0-4c4b-aaaa-659afa573e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:57.047599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dad63a-09e0-4c4b-aaaa-659afa573e30 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:57.068395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1038/2065] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:00:56 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:00:57.072999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a3fb7-f309-468c-af1a-ca31dc959087 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:57.083087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a3fb7-f309-468c-af1a-ca31dc959087 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:57.088604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567936e8-bbbd-409e-91b1-a64a5b40c2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:57.089395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567936e8-bbbd-409e-91b1-a64a5b40c2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:57.089675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567936e8-bbbd-409e-91b1-a64a5b40c2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:57.089976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567936e8-bbbd-409e-91b1-a64a5b40c2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:57.090606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567936e8-bbbd-409e-91b1-a64a5b40c2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:57.092206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567936e8-bbbd-409e-91b1-a64a5b40c2d2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:57.102465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1028/2066] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:00:57.106248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dad63a-09e0-4c4b-aaaa-659afa573e30 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:57.116770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dad63a-09e0-4c4b-aaaa-659afa573e30 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:57.121426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ee21a-8e85-475d-bce7-00f329183aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:57.122018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ee21a-8e85-475d-bce7-00f329183aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:57.122298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ee21a-8e85-475d-bce7-00f329183aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:57.122592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ee21a-8e85-475d-bce7-00f329183aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:57.123267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ee21a-8e85-475d-bce7-00f329183aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:57.124844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ee21a-8e85-475d-bce7-00f329183aa5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:57.135697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1039/2067] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:00:57.139702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567936e8-bbbd-409e-91b1-a64a5b40c2d2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:57.150205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567936e8-bbbd-409e-91b1-a64a5b40c2d2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:57.154840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:57.155501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:57.155778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:57.156043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] PATH_INFO: `/v3/projects/f2e5245da9a24e679c6e19089e9aa199/users/0b2c9c8c10ec455f85cd95a813692de5/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:57.171430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e5245da9a24e679c6e19089e9aa199, user_id=0b2c9c8c10ec455f85cd95a813692de5, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:57.174103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:57.188907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1029/2068] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:00:57 2022] PUT /identity/v3/projects/f2e5245da9a24e679c6e19089e9aa199/users/0b2c9c8c10ec455f85cd95a813692de5/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:00:57.193776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a90bb4-9263-486f-b4eb-919ebcf581f2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:57.194838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a90bb4-9263-486f-b4eb-919ebcf581f2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:57.195185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a90bb4-9263-486f-b4eb-919ebcf581f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:57.214311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-39a90bb4-9263-486f-b4eb-919ebcf581f2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:57.215547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-39a90bb4-9263-486f-b4eb-919ebcf581f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:57.263296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:57.276652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1040/2069] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:57.285785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:57.287535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:57.346681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f254d2-f5a2-43ef-bf82-8196adb0167f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:57.350322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce02777-08de-4a38-af9b-183f23f120ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:57.350959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02777-08de-4a38-af9b-183f23f120ce service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:57.351364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02777-08de-4a38-af9b-183f23f120ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:57.351563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02777-08de-4a38-af9b-183f23f120ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:57.360668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce02777-08de-4a38-af9b-183f23f120ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:57.362072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce02777-08de-4a38-af9b-183f23f120ce service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:57.372182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1030/2070] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:02.289762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6be784-aa10-4ab5-b730-c8a507f48fc0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:02.290335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6be784-aa10-4ab5-b730-c8a507f48fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:02.290777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6be784-aa10-4ab5-b730-c8a507f48fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:02.357681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-7c6be784-aa10-4ab5-b730-c8a507f48fc0 None None] MFA Rules not processed for user `0b2c9c8c10ec455f85cd95a813692de5`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:02.359380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7c6be784-aa10-4ab5-b730-c8a507f48fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:02.514613 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1041/2071] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:02.515779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce02777-08de-4a38-af9b-183f23f120ce service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:02.526299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce02777-08de-4a38-af9b-183f23f120ce service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:02.531263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b31b17-69ab-456a-8027-a05ada98e54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:02.532185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b31b17-69ab-456a-8027-a05ada98e54e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:02.532675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b31b17-69ab-456a-8027-a05ada98e54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:02.533297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b31b17-69ab-456a-8027-a05ada98e54e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:02.534000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b31b17-69ab-456a-8027-a05ada98e54e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:02.535999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b31b17-69ab-456a-8027-a05ada98e54e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:02.622125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1031/2072] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:02 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:02.626612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6be784-aa10-4ab5-b730-c8a507f48fc0 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:02.649111 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6be784-aa10-4ab5-b730-c8a507f48fc0 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:02.736075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dd999-34e3-4c9f-bd2c-fffc28266c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:02.736833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dd999-34e3-4c9f-bd2c-fffc28266c80 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:02.737233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dd999-34e3-4c9f-bd2c-fffc28266c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:02.737815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dd999-34e3-4c9f-bd2c-fffc28266c80 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:02.738500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dd999-34e3-4c9f-bd2c-fffc28266c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:02.741013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dd999-34e3-4c9f-bd2c-fffc28266c80 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:02.767921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1042/2073] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:02 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:01:02.771961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b31b17-69ab-456a-8027-a05ada98e54e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:02.783538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b31b17-69ab-456a-8027-a05ada98e54e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:02.788203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c5651-a2c9-4b6a-a04d-df623e31a18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:02.788809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c5651-a2c9-4b6a-a04d-df623e31a18c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:02.789095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c5651-a2c9-4b6a-a04d-df623e31a18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:02.789372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c5651-a2c9-4b6a-a04d-df623e31a18c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:02.790090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c5651-a2c9-4b6a-a04d-df623e31a18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:02.791711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c5651-a2c9-4b6a-a04d-df623e31a18c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:02.802373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1032/2074] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:02.807311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dd999-34e3-4c9f-bd2c-fffc28266c80 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:02.817266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dd999-34e3-4c9f-bd2c-fffc28266c80 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:02.822668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4c3fe-04f2-4fb2-b336-3e5a853bdae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:02.823612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4c3fe-04f2-4fb2-b336-3e5a853bdae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:02.823792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4c3fe-04f2-4fb2-b336-3e5a853bdae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:02.824069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4c3fe-04f2-4fb2-b336-3e5a853bdae2 admin admin] PATH_INFO: `/v3/projects/7bd636aa3db54e8c9eb398a31647292a/users/8fba495fbddf48ee833bdd4fb1fb48cf/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:02.841490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4c3fe-04f2-4fb2-b336-3e5a853bdae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd636aa3db54e8c9eb398a31647292a, user_id=8fba495fbddf48ee833bdd4fb1fb48cf, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:02.844380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4c3fe-04f2-4fb2-b336-3e5a853bdae2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:02.860761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1043/2075] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:02 2022] PUT /identity/v3/projects/7bd636aa3db54e8c9eb398a31647292a/users/8fba495fbddf48ee833bdd4fb1fb48cf/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:02.864956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c5651-a2c9-4b6a-a04d-df623e31a18c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:02.876660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c5651-a2c9-4b6a-a04d-df623e31a18c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:02.938649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:02.939289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:02.939570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:02.940008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:02.940646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:02.942214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:02.953465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1033/2076] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:02.957657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4c3fe-04f2-4fb2-b336-3e5a853bdae2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:02.967738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4c3fe-04f2-4fb2-b336-3e5a853bdae2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:02.972645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4b864-0ae6-48cd-8aeb-60eedf2ed666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:02.973177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4b864-0ae6-48cd-8aeb-60eedf2ed666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:02.973327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4b864-0ae6-48cd-8aeb-60eedf2ed666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:02.973600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4b864-0ae6-48cd-8aeb-60eedf2ed666 admin admin] PATH_INFO: `/v3/projects/7bd636aa3db54e8c9eb398a31647292a/users/8fba495fbddf48ee833bdd4fb1fb48cf/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:02.975544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4b864-0ae6-48cd-8aeb-60eedf2ed666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd636aa3db54e8c9eb398a31647292a, user_id=8fba495fbddf48ee833bdd4fb1fb48cf, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:02.978751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4b864-0ae6-48cd-8aeb-60eedf2ed666 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:02.991007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1044/2077] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:02 2022] PUT /identity/v3/projects/7bd636aa3db54e8c9eb398a31647292a/users/8fba495fbddf48ee833bdd4fb1fb48cf/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:03.221303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:03.223145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:03.236114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432ee18-10a3-4bcc-b700-18ba769026d3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:03.257809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:03.316462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:03.317109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:03.317388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:03.317655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] PATH_INFO: `/v3/users/f286272b22da470f942aba726058c626` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:03.318820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f286272b22da470f942aba726058c626)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:03.320859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:03.392511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f286272b22da470f942aba726058c626'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:03.401576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f286272b22da470f942aba726058c626'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:03.402364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f286272b22da470f942aba726058c626'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:03.403207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f286272b22da470f942aba726058c626'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:03.416650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f286272b22da470f942aba726058c626'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:03.442924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1034/2078] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:01:03 2022] DELETE /identity/v3/users/f286272b22da470f942aba726058c626 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:03.448286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e965e9d-2e99-4c50-8458-d3a7f560dbd1 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:03.448839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e965e9d-2e99-4c50-8458-d3a7f560dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:03.448929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e965e9d-2e99-4c50-8458-d3a7f560dbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:03.481622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6e965e9d-2e99-4c50-8458-d3a7f560dbd1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:03.484298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6e965e9d-2e99-4c50-8458-d3a7f560dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:03.529278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:03.578327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1045/2079] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:03.590241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:03.591877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:03.692239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12de2f-af9c-437b-95d3-25e43cb3e624 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:03.702628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:03.703568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:03.704075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:03.704476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:03.721882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:03.723502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:03.763039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1035/2080] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:04.120303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e965e9d-2e99-4c50-8458-d3a7f560dbd1 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:04.121763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6e965e9d-2e99-4c50-8458-d3a7f560dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:04.136652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e965e9d-2e99-4c50-8458-d3a7f560dbd1 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:04.142052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:04.142920 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:04.143325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:04.143684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] PATH_INFO: `/v3/projects/c3c9f5154d5e4f61aa9f2d731f1c772d` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:04.145703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c9f5154d5e4f61aa9f2d731f1c772d)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:04.150131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:04.279205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c9f5154d5e4f61aa9f2d731f1c772d', 'request_id': 'req-df8337a5-8228-43cf-9310-db2bc1a7782b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:04.301505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c9f5154d5e4f61aa9f2d731f1c772d', 'request_id': 'req-df8337a5-8228-43cf-9310-db2bc1a7782b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:04.312153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c9f5154d5e4f61aa9f2d731f1c772d', 'request_id': 'req-df8337a5-8228-43cf-9310-db2bc1a7782b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:04.327333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] The token cache is being invalidate because project c3c9f5154d5e4f61aa9f2d731f1c772d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:01:04.327823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:04.328913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:04.331150 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1046/2081] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:01:04 2022] DELETE /identity/v3/projects/c3c9f5154d5e4f61aa9f2d731f1c772d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:07.367848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:07.370313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:07.389398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160c87-a15a-42d0-93f2-81db9434f948 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:07.471984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:07.497977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:07.498820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:07.500410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:07.500754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] PATH_INFO: `/v3/users/2c92e7453b1645b29c6c8ad7c38b7735` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:07.502666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c92e7453b1645b29c6c8ad7c38b7735)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:07.505024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:07.581534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c92e7453b1645b29c6c8ad7c38b7735'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:07.595756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c92e7453b1645b29c6c8ad7c38b7735'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:07.596583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c92e7453b1645b29c6c8ad7c38b7735'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:07.597206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c92e7453b1645b29c6c8ad7c38b7735'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:07.607533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5653ba2f-6f32-4f82-bc6e-fadb5f608281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c92e7453b1645b29c6c8ad7c38b7735'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:07.639798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1036/2082] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:01:07 2022] DELETE /identity/v3/users/2c92e7453b1645b29c6c8ad7c38b7735 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:07.972341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:07.974030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:07.998269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8337a5-8228-43cf-9310-db2bc1a7782b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:08.106174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:08.107066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:08.107419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:08.108122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] PATH_INFO: `/v3/projects/45628e5deeb045e4bdf7bc3b53555d67` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:08.109834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=45628e5deeb045e4bdf7bc3b53555d67)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:08.112690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:08.249320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45628e5deeb045e4bdf7bc3b53555d67', 'request_id': 'req-f649eac5-5d68-4d94-bc55-2701fa91bb6c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.266615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45628e5deeb045e4bdf7bc3b53555d67', 'request_id': 'req-f649eac5-5d68-4d94-bc55-2701fa91bb6c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.277927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45628e5deeb045e4bdf7bc3b53555d67', 'request_id': 'req-f649eac5-5d68-4d94-bc55-2701fa91bb6c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.280275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:08.287781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1de4b-d701-4962-8f14-79245bdf9f61 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:08.288263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1de4b-d701-4962-8f14-79245bdf9f61 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:08.288646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1de4b-d701-4962-8f14-79245bdf9f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:08.300742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] The token cache is being invalidate because project 45628e5deeb045e4bdf7bc3b53555d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:01:08.301062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.301736 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.304198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1047/2083] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:01:07 2022] DELETE /identity/v3/projects/45628e5deeb045e4bdf7bc3b53555d67 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:08.309490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:08.311256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:08.326383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-10f1de4b-d701-4962-8f14-79245bdf9f61 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:08.329110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10f1de4b-d701-4962-8f14-79245bdf9f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:08.421190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649eac5-5d68-4d94-bc55-2701fa91bb6c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:08.426602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:08.427491 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:08.428195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:08.428624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] PATH_INFO: `/v3/users/9da6cc2bfc99435296c186a10f89b5f6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:08.430157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9da6cc2bfc99435296c186a10f89b5f6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:08.432448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1037/2084] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:01:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:08.445647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:08.489962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1de4b-d701-4962-8f14-79245bdf9f61 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:08.491710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10f1de4b-d701-4962-8f14-79245bdf9f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:08.541050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9da6cc2bfc99435296c186a10f89b5f6'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.557927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9da6cc2bfc99435296c186a10f89b5f6'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.559542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9da6cc2bfc99435296c186a10f89b5f6'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.560475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9da6cc2bfc99435296c186a10f89b5f6'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.577798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9da6cc2bfc99435296c186a10f89b5f6'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:08.616904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1048/2085] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:01:08 2022] DELETE /identity/v3/users/9da6cc2bfc99435296c186a10f89b5f6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:01:08.656195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1de4b-d701-4962-8f14-79245bdf9f61 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:08.665179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:08.747859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45fae7-767a-4bd9-a2e0-24ba036b8867 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:08.748966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45fae7-767a-4bd9-a2e0-24ba036b8867 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:08.750173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45fae7-767a-4bd9-a2e0-24ba036b8867 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:08.750526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45fae7-767a-4bd9-a2e0-24ba036b8867 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:08.757189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cc45fae7-767a-4bd9-a2e0-24ba036b8867 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:08.844779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45fae7-767a-4bd9-a2e0-24ba036b8867 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:08.846486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45fae7-767a-4bd9-a2e0-24ba036b8867 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:08.895636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1038/2086] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:01:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 12:01:09.083068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:09.091090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:09.290603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640820c5-013c-4d1c-92de-ca4f9a4e76ce admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:09.318605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:09.319326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:09.319632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:09.320429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] PATH_INFO: `/v3/projects/50cb17ecff894cbe800d59cfef6042ac` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:09.321722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50cb17ecff894cbe800d59cfef6042ac)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:09.323839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:09.362244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:09.454760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50cb17ecff894cbe800d59cfef6042ac', 'request_id': 'req-e70a4737-8571-4fbf-83e3-14258bf38d82'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:09.469951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50cb17ecff894cbe800d59cfef6042ac', 'request_id': 'req-e70a4737-8571-4fbf-83e3-14258bf38d82'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:09.480155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50cb17ecff894cbe800d59cfef6042ac', 'request_id': 'req-e70a4737-8571-4fbf-83e3-14258bf38d82'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:09.496605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] The token cache is being invalidate because project 50cb17ecff894cbe800d59cfef6042ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:01:09.497028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:09.498158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e70a4737-8571-4fbf-83e3-14258bf38d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:09.500136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1049/2087] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:01:09 2022] DELETE /identity/v3/projects/50cb17ecff894cbe800d59cfef6042ac => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:09.505985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8baa32-1e72-4def-bcef-9964b4448312 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:09.509672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8baa32-1e72-4def-bcef-9964b4448312 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:09.510482 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8baa32-1e72-4def-bcef-9964b4448312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:09.560524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6e8baa32-1e72-4def-bcef-9964b4448312 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:09.562265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6e8baa32-1e72-4def-bcef-9964b4448312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:09.725886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1039/2088] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:09.734137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767151d-9e64-4289-948b-d52cb83ff124 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:09.734652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767151d-9e64-4289-948b-d52cb83ff124 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:09.735042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767151d-9e64-4289-948b-d52cb83ff124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:09.766988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e767151d-9e64-4289-948b-d52cb83ff124 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:09.768426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e767151d-9e64-4289-948b-d52cb83ff124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:09.802429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1050/2089] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:09.803947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8baa32-1e72-4def-bcef-9964b4448312 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:09.834074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8baa32-1e72-4def-bcef-9964b4448312 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:09.843646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:09.844281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:09.844569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:09.844891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:09.845686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:01:09.845991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:09.849133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:09.874729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1040/2090] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:01:09.876042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767151d-9e64-4289-948b-d52cb83ff124 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:09.891856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767151d-9e64-4289-948b-d52cb83ff124 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:09.898256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7919f81-e196-4f19-9a93-821df6ea2945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:09.898725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919f81-e196-4f19-9a93-821df6ea2945 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:09.899052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919f81-e196-4f19-9a93-821df6ea2945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:09.900016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919f81-e196-4f19-9a93-821df6ea2945 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:09.900905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7919f81-e196-4f19-9a93-821df6ea2945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:09.903179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7919f81-e196-4f19-9a93-821df6ea2945 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:09.975364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1051/2091] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:09 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:09.978033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:09.997378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c03c53-d5a0-4552-8cca-c2ad524b52a1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:10.136885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06effb-c4c9-4594-8e44-cfe1a6c54f96 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.137405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06effb-c4c9-4594-8e44-cfe1a6c54f96 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.137405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06effb-c4c9-4594-8e44-cfe1a6c54f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.141028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:10.177908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-fd06effb-c4c9-4594-8e44-cfe1a6c54f96 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:10.180137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fd06effb-c4c9-4594-8e44-cfe1a6c54f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.189031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf21339-58aa-46d1-a47b-4a9a2b034cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:10.189900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf21339-58aa-46d1-a47b-4a9a2b034cff admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.190349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf21339-58aa-46d1-a47b-4a9a2b034cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.190677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf21339-58aa-46d1-a47b-4a9a2b034cff admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.191736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf21339-58aa-46d1-a47b-4a9a2b034cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:10.195929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf21339-58aa-46d1-a47b-4a9a2b034cff admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:10.242182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1041/2092] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:09 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:10.244962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf21339-58aa-46d1-a47b-4a9a2b034cff admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:10.270028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf21339-58aa-46d1-a47b-4a9a2b034cff admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:10.283492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:10.284069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.284358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.284628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.285279 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:10.286927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:10.304195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1052/2093] 10.222.0.52 () {58 vars in 967 bytes} [Mon Oct 10 12:01:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:10.322868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1042/2094] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:10.326881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06effb-c4c9-4594-8e44-cfe1a6c54f96 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:10.329337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:10.332413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.342076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06effb-c4c9-4594-8e44-cfe1a6c54f96 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:10.351932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ab820-fecf-4a9e-bf9c-9a211d932232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:10.352605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab820-fecf-4a9e-bf9c-9a211d932232 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.353006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab820-fecf-4a9e-bf9c-9a211d932232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.353150 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab820-fecf-4a9e-bf9c-9a211d932232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.354368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ab820-fecf-4a9e-bf9c-9a211d932232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:10.358022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ab820-fecf-4a9e-bf9c-9a211d932232 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:10.369960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1053/2095] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:10.374405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dee2c9-85a3-452c-ae57-68015948ed2b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.374874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dee2c9-85a3-452c-ae57-68015948ed2b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.375312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dee2c9-85a3-452c-ae57-68015948ed2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.416380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-86dee2c9-85a3-452c-ae57-68015948ed2b None None] MFA Rules not processed for user `8fba495fbddf48ee833bdd4fb1fb48cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:10.425202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-86dee2c9-85a3-452c-ae57-68015948ed2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.456507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d452fc-8140-49a9-bf8d-549d10b3ea2c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:10.463082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965ffad-cee9-4485-9659-883f3ef5038a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:10.463703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965ffad-cee9-4485-9659-883f3ef5038a service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.463984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965ffad-cee9-4485-9659-883f3ef5038a service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.464453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965ffad-cee9-4485-9659-883f3ef5038a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.474521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965ffad-cee9-4485-9659-883f3ef5038a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:10.476480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965ffad-cee9-4485-9659-883f3ef5038a service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:10.485583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1043/2096] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:01:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:01:10.486200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965ffad-cee9-4485-9659-883f3ef5038a service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:10.497468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965ffad-cee9-4485-9659-883f3ef5038a service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:10.513502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991f4a7-412a-4518-b3a0-3e2ffb603643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:10.514184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4a7-412a-4518-b3a0-3e2ffb603643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.514729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4a7-412a-4518-b3a0-3e2ffb603643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.514729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991f4a7-412a-4518-b3a0-3e2ffb603643 admin admin] PATH_INFO: `/v3/projects/dbe02594043f458bbe4336fb7d491b7f/users/0fb8f1e814e841abaeec890cc8865504/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.520274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:10.536658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991f4a7-412a-4518-b3a0-3e2ffb603643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe02594043f458bbe4336fb7d491b7f, user_id=0fb8f1e814e841abaeec890cc8865504, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:10.550300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991f4a7-412a-4518-b3a0-3e2ffb603643 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:10.556156 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1054/2097] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:10.576396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dee2c9-85a3-452c-ae57-68015948ed2b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:10.580444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-86dee2c9-85a3-452c-ae57-68015948ed2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.583984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1044/2098] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:10 2022] PUT /identity/v3/projects/dbe02594043f458bbe4336fb7d491b7f/users/0fb8f1e814e841abaeec890cc8865504/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:10.586534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45268f1-3507-4e77-8de4-cd2c4a86499f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.587591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45268f1-3507-4e77-8de4-cd2c4a86499f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.588153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45268f1-3507-4e77-8de4-cd2c4a86499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.654131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e45268f1-3507-4e77-8de4-cd2c4a86499f None None] MFA Rules not processed for user `0fb8f1e814e841abaeec890cc8865504`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:10.656590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e45268f1-3507-4e77-8de4-cd2c4a86499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.701136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dee2c9-85a3-452c-ae57-68015948ed2b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:10.803638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1045/2099] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:10.817438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45268f1-3507-4e77-8de4-cd2c4a86499f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:10.819690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e45268f1-3507-4e77-8de4-cd2c4a86499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.851337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd0502-6e75-4441-bc2d-94cc8059b22e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6774fdd8b2e46b6a7d00919568f9fd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:10.854987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0502-6e75-4441-bc2d-94cc8059b22e service glance] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.862780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0502-6e75-4441-bc2d-94cc8059b22e service glance] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.864525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0502-6e75-4441-bc2d-94cc8059b22e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.876928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-59cd0502-6e75-4441-bc2d-94cc8059b22e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.892429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:10.990590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd0502-6e75-4441-bc2d-94cc8059b22e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:10.992275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd0502-6e75-4441-bc2d-94cc8059b22e service glance] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:11.004772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45268f1-3507-4e77-8de4-cd2c4a86499f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:11.009623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1055/2100] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:11.012508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81c9d5-5000-428b-9c20-fca8060cf8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:11.013206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81c9d5-5000-428b-9c20-fca8060cf8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:11.013550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81c9d5-5000-428b-9c20-fca8060cf8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:11.013884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81c9d5-5000-428b-9c20-fca8060cf8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:11.025710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81c9d5-5000-428b-9c20-fca8060cf8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:11.027702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81c9d5-5000-428b-9c20-fca8060cf8c8 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:11.032226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f041478-e43b-42e1-a74f-7c139ebde7a0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:11.032584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f041478-e43b-42e1-a74f-7c139ebde7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:11.032927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f041478-e43b-42e1-a74f-7c139ebde7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:11.058885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1046/2101] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:11.066010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3f041478-e43b-42e1-a74f-7c139ebde7a0 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:11.067678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3f041478-e43b-42e1-a74f-7c139ebde7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:11.146492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1056/2102] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:01:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:11.164396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81c9d5-5000-428b-9c20-fca8060cf8c8 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:11.244741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347319ed-0aa9-4415-a232-f4f18cce501c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:11.245033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347319ed-0aa9-4415-a232-f4f18cce501c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:11.245309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347319ed-0aa9-4415-a232-f4f18cce501c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:11.262644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:11.275178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-347319ed-0aa9-4415-a232-f4f18cce501c None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:11.276416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-347319ed-0aa9-4415-a232-f4f18cce501c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:11.287739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81c9d5-5000-428b-9c20-fca8060cf8c8 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:11.294661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ea56e-c1cc-4edd-afdc-abfb25cb4f96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:11.295504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ea56e-c1cc-4edd-afdc-abfb25cb4f96 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:11.296072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ea56e-c1cc-4edd-afdc-abfb25cb4f96 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:11.296792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ea56e-c1cc-4edd-afdc-abfb25cb4f96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:11.306784 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1057/2103] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:01:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:11.309023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ea56e-c1cc-4edd-afdc-abfb25cb4f96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:11.310617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ea56e-c1cc-4edd-afdc-abfb25cb4f96 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:11.320148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1047/2104] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:01:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:01:11.326867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347319ed-0aa9-4415-a232-f4f18cce501c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:11.359318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347319ed-0aa9-4415-a232-f4f18cce501c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:11.365083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82819710-a55d-4f68-ad06-32cb1a0d17d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:11.365761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82819710-a55d-4f68-ad06-32cb1a0d17d3 service swift] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:11.366039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82819710-a55d-4f68-ad06-32cb1a0d17d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:11.366510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82819710-a55d-4f68-ad06-32cb1a0d17d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:11.379073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82819710-a55d-4f68-ad06-32cb1a0d17d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:11.381098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82819710-a55d-4f68-ad06-32cb1a0d17d3 service swift] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:11.399289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1058/2105] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:01:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:01:12.232821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b5173-abac-4c4e-9408-7c924ba5b8c6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:12.233615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b5173-abac-4c4e-9408-7c924ba5b8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:12.234108 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b5173-abac-4c4e-9408-7c924ba5b8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:12.257546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ba5b5173-abac-4c4e-9408-7c924ba5b8c6 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:12.259399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ba5b5173-abac-4c4e-9408-7c924ba5b8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:12.269643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1048/2106] 10.222.0.52 () {58 vars in 967 bytes} [Mon Oct 10 12:01:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:12.293193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82819710-a55d-4f68-ad06-32cb1a0d17d3 service swift] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:12.305707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82819710-a55d-4f68-ad06-32cb1a0d17d3 service swift] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:12.316592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55248dde-998a-469c-b493-c6694aaebc87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:12.317714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55248dde-998a-469c-b493-c6694aaebc87 service swift] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:12.318195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55248dde-998a-469c-b493-c6694aaebc87 service swift] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:12.318640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55248dde-998a-469c-b493-c6694aaebc87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:12.340493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55248dde-998a-469c-b493-c6694aaebc87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:12.342354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55248dde-998a-469c-b493-c6694aaebc87 service swift] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:12.353234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1059/2107] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:01:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:01:17.541884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b5173-abac-4c4e-9408-7c924ba5b8c6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:17.544198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ba5b5173-abac-4c4e-9408-7c924ba5b8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:17.762797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b5173-abac-4c4e-9408-7c924ba5b8c6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:17.777925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:17.780902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:17.781194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:17.781476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] PATH_INFO: `/v3/users/ef293aceb5e7483f9d69a4eab9e187fa` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:17.782834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef293aceb5e7483f9d69a4eab9e187fa)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:17.785028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:17.876916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:17.880730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef293aceb5e7483f9d69a4eab9e187fa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:17.894825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef293aceb5e7483f9d69a4eab9e187fa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:17.895664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef293aceb5e7483f9d69a4eab9e187fa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:17.896285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef293aceb5e7483f9d69a4eab9e187fa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:17.909529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef293aceb5e7483f9d69a4eab9e187fa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:17.937761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1049/2108] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:01:17 2022] DELETE /identity/v3/users/ef293aceb5e7483f9d69a4eab9e187fa => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:17.942704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397ed1a-b6b7-47d5-84ec-b440c6bcdfb9 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:17.943242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397ed1a-b6b7-47d5-84ec-b440c6bcdfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:17.943396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397ed1a-b6b7-47d5-84ec-b440c6bcdfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:17.973629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5397ed1a-b6b7-47d5-84ec-b440c6bcdfb9 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:17.975190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5397ed1a-b6b7-47d5-84ec-b440c6bcdfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:18.084295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1060/2109] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:18.097210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:18.099371 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:18.217921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8787a2-d675-406d-b164-814cb855da5e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:18.234305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902839e-15a4-49b7-abbf-294ac3d55a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:18.235136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902839e-15a4-49b7-abbf-294ac3d55a01 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:18.235727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902839e-15a4-49b7-abbf-294ac3d55a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:18.235974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902839e-15a4-49b7-abbf-294ac3d55a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:18.254751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902839e-15a4-49b7-abbf-294ac3d55a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:18.258922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902839e-15a4-49b7-abbf-294ac3d55a01 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:18.286474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1050/2110] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:18.641402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397ed1a-b6b7-47d5-84ec-b440c6bcdfb9 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:18.644484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5397ed1a-b6b7-47d5-84ec-b440c6bcdfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:18.680128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397ed1a-b6b7-47d5-84ec-b440c6bcdfb9 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:18.687637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:18.689025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:18.689386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:18.689794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] PATH_INFO: `/v3/projects/132383472a4f48779d8c4fc5a5179a8e` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:18.691868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=132383472a4f48779d8c4fc5a5179a8e)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:18.693614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:18.763153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132383472a4f48779d8c4fc5a5179a8e', 'request_id': 'req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:18.775572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132383472a4f48779d8c4fc5a5179a8e', 'request_id': 'req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:18.785357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132383472a4f48779d8c4fc5a5179a8e', 'request_id': 'req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:18.803354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] The token cache is being invalidate because project 132383472a4f48779d8c4fc5a5179a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:01:18.803550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:18.804270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-54ed1872-5163-4f51-b8c7-755cdc7b5d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:18.810937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1061/2111] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:01:18 2022] DELETE /identity/v3/projects/132383472a4f48779d8c4fc5a5179a8e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:18.813433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950c35-77df-4394-88a1-76e44b886cc0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:18.813890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950c35-77df-4394-88a1-76e44b886cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:18.814007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950c35-77df-4394-88a1-76e44b886cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:18.837808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-70950c35-77df-4394-88a1-76e44b886cc0 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:18.839446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-70950c35-77df-4394-88a1-76e44b886cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:18.971640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1051/2112] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:18.983667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641bdb-8b73-4e4f-aba5-42d1e2fae531 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:18.983667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641bdb-8b73-4e4f-aba5-42d1e2fae531 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:18.983667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641bdb-8b73-4e4f-aba5-42d1e2fae531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:18.998343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-26641bdb-8b73-4e4f-aba5-42d1e2fae531 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:18.999797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-26641bdb-8b73-4e4f-aba5-42d1e2fae531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:19.025616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1062/2113] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:19.026994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950c35-77df-4394-88a1-76e44b886cc0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:19.046188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950c35-77df-4394-88a1-76e44b886cc0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:19.050484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:19.051157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.051444 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.051717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.052384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:01:19.052686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:19.054664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:19.086453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1052/2114] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:01:19.089134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26641bdb-8b73-4e4f-aba5-42d1e2fae531 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:19.102396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26641bdb-8b73-4e4f-aba5-42d1e2fae531 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:19.126758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833a2e0-cfe1-4bef-94b6-46c33c1dbfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:19.127524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a2e0-cfe1-4bef-94b6-46c33c1dbfd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.127818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a2e0-cfe1-4bef-94b6-46c33c1dbfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.128094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a2e0-cfe1-4bef-94b6-46c33c1dbfd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.128741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833a2e0-cfe1-4bef-94b6-46c33c1dbfd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:19.132039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833a2e0-cfe1-4bef-94b6-46c33c1dbfd2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:19.222353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1063/2115] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:19 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:19.226014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:19.236944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974e269-9942-4543-bd44-67b90c23e286 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:19.297912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:19.313719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdf815-76d4-4a28-869d-3dc4f44b880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:19.314547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf815-76d4-4a28-869d-3dc4f44b880a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.314959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf815-76d4-4a28-869d-3dc4f44b880a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.315471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdf815-76d4-4a28-869d-3dc4f44b880a admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.316224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cdf815-76d4-4a28-869d-3dc4f44b880a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:19.319774 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cdf815-76d4-4a28-869d-3dc4f44b880a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:19.353164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1053/2116] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:19 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:19.357996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833a2e0-cfe1-4bef-94b6-46c33c1dbfd2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:19.373869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833a2e0-cfe1-4bef-94b6-46c33c1dbfd2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:19.380312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82045a-3f7c-4b71-984f-396887749a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:19.381052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82045a-3f7c-4b71-984f-396887749a4b admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.381496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82045a-3f7c-4b71-984f-396887749a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.381864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82045a-3f7c-4b71-984f-396887749a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.384298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82045a-3f7c-4b71-984f-396887749a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:19.386326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82045a-3f7c-4b71-984f-396887749a4b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:19.406009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdf815-76d4-4a28-869d-3dc4f44b880a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:19.406525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1064/2117] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:19.421194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdf815-76d4-4a28-869d-3dc4f44b880a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:19.437813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f1859-d998-4091-9c49-9de7080ad632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:19.438445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f1859-d998-4091-9c49-9de7080ad632 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.438731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f1859-d998-4091-9c49-9de7080ad632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.439029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f1859-d998-4091-9c49-9de7080ad632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.439677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f1859-d998-4091-9c49-9de7080ad632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:19.441271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f1859-d998-4091-9c49-9de7080ad632 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:19.456088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1054/2118] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:01:19.459914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82045a-3f7c-4b71-984f-396887749a4b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:19.471948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82045a-3f7c-4b71-984f-396887749a4b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:19.478003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:19.479086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.479579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.480074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] PATH_INFO: `/v3/projects/638435e7c9ea431aa6ad3674ee6ff2ae/users/ce6a1b5ca7584f90bfe83b73aff1335a/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.497743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:19.501843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=638435e7c9ea431aa6ad3674ee6ff2ae, user_id=ce6a1b5ca7584f90bfe83b73aff1335a, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:19.506464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:19.530730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1065/2119] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:19 2022] PUT /identity/v3/projects/638435e7c9ea431aa6ad3674ee6ff2ae/users/ce6a1b5ca7584f90bfe83b73aff1335a/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:01:19.535581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf831d1-0cb5-4414-a0cd-61cd8d36d884 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.536024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf831d1-0cb5-4414-a0cd-61cd8d36d884 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.536372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf831d1-0cb5-4414-a0cd-61cd8d36d884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.564542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cbf831d1-0cb5-4414-a0cd-61cd8d36d884 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:19.565923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cbf831d1-0cb5-4414-a0cd-61cd8d36d884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:19.714067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1055/2120] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:19.731080 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:19.733883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:19.744703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff44c8-68d3-4154-a9c5-d433f105a720 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.745256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff44c8-68d3-4154-a9c5-d433f105a720 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.745887 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff44c8-68d3-4154-a9c5-d433f105a720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.805206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2aff44c8-68d3-4154-a9c5-d433f105a720 None None] MFA Rules not processed for user `2f7735444ca8431ea7906070374dbe67`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:19.807943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2aff44c8-68d3-4154-a9c5-d433f105a720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:19.876333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3a25a-9417-4aff-96f6-3eeac641df85 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:19.881065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3376015-7bc1-40d7-8f30-67136f309986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:19.881970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3376015-7bc1-40d7-8f30-67136f309986 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:19.882429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3376015-7bc1-40d7-8f30-67136f309986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:19.882854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3376015-7bc1-40d7-8f30-67136f309986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:19.894575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3376015-7bc1-40d7-8f30-67136f309986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:19.896499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3376015-7bc1-40d7-8f30-67136f309986 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:19.907532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1056/2121] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:19.908699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1066/2122] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:26.204729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a79f5-152d-4a59-acdf-a4ca448f79f0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:26.205297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a79f5-152d-4a59-acdf-a4ca448f79f0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:26.206059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a79f5-152d-4a59-acdf-a4ca448f79f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:26.238023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-716a79f5-152d-4a59-acdf-a4ca448f79f0 None None] MFA Rules not processed for user `0b2c9c8c10ec455f85cd95a813692de5`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:26.240562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-716a79f5-152d-4a59-acdf-a4ca448f79f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:26.346475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1057/2123] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:26.598723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0743fd-c2fd-4ee5-bc4d-f2f91a2def98 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:26.599684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0743fd-c2fd-4ee5-bc4d-f2f91a2def98 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:26.600035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0743fd-c2fd-4ee5-bc4d-f2f91a2def98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:26.631184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5d0743fd-c2fd-4ee5-bc4d-f2f91a2def98 None None] MFA Rules not processed for user `ce6a1b5ca7584f90bfe83b73aff1335a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:26.632557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5d0743fd-c2fd-4ee5-bc4d-f2f91a2def98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:26.719303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1067/2124] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:26.723399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a79f5-152d-4a59-acdf-a4ca448f79f0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:26.741379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a79f5-152d-4a59-acdf-a4ca448f79f0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:26.746556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35016e-33ae-4ec3-9604-1df8518a6b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:26.747199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35016e-33ae-4ec3-9604-1df8518a6b2e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:26.747476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35016e-33ae-4ec3-9604-1df8518a6b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:26.747746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35016e-33ae-4ec3-9604-1df8518a6b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:26.748448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35016e-33ae-4ec3-9604-1df8518a6b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:26.750349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35016e-33ae-4ec3-9604-1df8518a6b2e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:26.777472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:26.790701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1058/2125] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:26 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:26.791747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0743fd-c2fd-4ee5-bc4d-f2f91a2def98 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:26.804054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0743fd-c2fd-4ee5-bc4d-f2f91a2def98 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:26.901353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab6492-a937-4a86-8275-d1f9d01ae1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:26.901960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab6492-a937-4a86-8275-d1f9d01ae1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:26.902238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab6492-a937-4a86-8275-d1f9d01ae1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:26.902505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab6492-a937-4a86-8275-d1f9d01ae1d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:26.903166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab6492-a937-4a86-8275-d1f9d01ae1d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:26.912448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab6492-a937-4a86-8275-d1f9d01ae1d5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:26.947286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1068/2126] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:26 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:01:26.948855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35016e-33ae-4ec3-9604-1df8518a6b2e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:26.960256 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35016e-33ae-4ec3-9604-1df8518a6b2e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:26.965583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08292187-0e57-4c54-8faf-d12de38621b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:26.966266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08292187-0e57-4c54-8faf-d12de38621b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:26.966611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08292187-0e57-4c54-8faf-d12de38621b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:26.966978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08292187-0e57-4c54-8faf-d12de38621b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:26.967681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08292187-0e57-4c54-8faf-d12de38621b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:26.969378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08292187-0e57-4c54-8faf-d12de38621b5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:26.980788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1059/2127] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:26.984976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab6492-a937-4a86-8275-d1f9d01ae1d5 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:26.998163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab6492-a937-4a86-8275-d1f9d01ae1d5 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:27.003499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335d6f-b39a-4317-bd17-3fab4811c939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:27.004174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335d6f-b39a-4317-bd17-3fab4811c939 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:27.004456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335d6f-b39a-4317-bd17-3fab4811c939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:27.004728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335d6f-b39a-4317-bd17-3fab4811c939 admin admin] PATH_INFO: `/v3/projects/53dfa853fdbd4749a0521835994ce87b/users/06d7f6b94e574f1a89c4c41b55d79044/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:27.017729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:27.022664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335d6f-b39a-4317-bd17-3fab4811c939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53dfa853fdbd4749a0521835994ce87b, user_id=06d7f6b94e574f1a89c4c41b55d79044, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:27.025632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335d6f-b39a-4317-bd17-3fab4811c939 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:27.040769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1069/2128] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:26 2022] PUT /identity/v3/projects/53dfa853fdbd4749a0521835994ce87b/users/06d7f6b94e574f1a89c4c41b55d79044/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:27.044846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08292187-0e57-4c54-8faf-d12de38621b5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:27.057337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08292187-0e57-4c54-8faf-d12de38621b5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:27.147197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b7735-ca48-446d-8dbb-b92f99a9927f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:27.148375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b7735-ca48-446d-8dbb-b92f99a9927f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:27.148795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b7735-ca48-446d-8dbb-b92f99a9927f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:27.149124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b7735-ca48-446d-8dbb-b92f99a9927f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:27.149880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b7735-ca48-446d-8dbb-b92f99a9927f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:27.151837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b7735-ca48-446d-8dbb-b92f99a9927f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:27.169584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1060/2129] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:01:27.173577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335d6f-b39a-4317-bd17-3fab4811c939 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:27.190829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335d6f-b39a-4317-bd17-3fab4811c939 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:27.197908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:27.198618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:27.198923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:27.199211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] PATH_INFO: `/v3/projects/53dfa853fdbd4749a0521835994ce87b/users/06d7f6b94e574f1a89c4c41b55d79044/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:27.201424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] RBAC: Authorizing `identity:create_grant(project_id=53dfa853fdbd4749a0521835994ce87b, user_id=06d7f6b94e574f1a89c4c41b55d79044, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:27.205118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:27.218565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1070/2130] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:27 2022] PUT /identity/v3/projects/53dfa853fdbd4749a0521835994ce87b/users/06d7f6b94e574f1a89c4c41b55d79044/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:33.111137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d840-eb65-49c5-9393-f31e831f9430 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:33.112120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d840-eb65-49c5-9393-f31e831f9430 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:33.112691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d840-eb65-49c5-9393-f31e831f9430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:33.143028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e6a1d840-eb65-49c5-9393-f31e831f9430 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:33.145362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6a1d840-eb65-49c5-9393-f31e831f9430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:33.270517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1061/2131] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:33.285860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:33.288157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:33.335640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1d840-eb65-49c5-9393-f31e831f9430 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:33.337941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6a1d840-eb65-49c5-9393-f31e831f9430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:33.421240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c7bf5-1bae-4f9e-b237-54b06c56468c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:33.426587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587d58d-c27d-440d-8b85-3affa3677d73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:33.427392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d58d-c27d-440d-8b85-3affa3677d73 service swift] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:33.428038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d58d-c27d-440d-8b85-3affa3677d73 service swift] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:33.428407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587d58d-c27d-440d-8b85-3affa3677d73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:33.438146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587d58d-c27d-440d-8b85-3affa3677d73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:33.441046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587d58d-c27d-440d-8b85-3affa3677d73 service swift] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:33.460305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1071/2132] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:01:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:01:33.472384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1d840-eb65-49c5-9393-f31e831f9430 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:33.478352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:33.479033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:33.479315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:33.480343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] PATH_INFO: `/v3/users/0fb8f1e814e841abaeec890cc8865504` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:33.481708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb8f1e814e841abaeec890cc8865504)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:33.484644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:33.566722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:33.594703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb8f1e814e841abaeec890cc8865504'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:33.621185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb8f1e814e841abaeec890cc8865504'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:33.622504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb8f1e814e841abaeec890cc8865504'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:33.623238 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb8f1e814e841abaeec890cc8865504'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:33.636298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb8f1e814e841abaeec890cc8865504'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:33.683638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1062/2133] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:01:33 2022] DELETE /identity/v3/users/0fb8f1e814e841abaeec890cc8865504 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:01:33.684156 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857914aa-8883-4e47-b492-537304f58b83 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:33.684443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857914aa-8883-4e47-b492-537304f58b83 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:33.684718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857914aa-8883-4e47-b492-537304f58b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:33.720457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-857914aa-8883-4e47-b492-537304f58b83 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:33.721689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-857914aa-8883-4e47-b492-537304f58b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:33.843170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1072/2134] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:33.881790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:33.884659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:34.019280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed487aa1-7a06-404d-b67f-16107f4eb9ff admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:34.035604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830815e-c8f3-447a-9d22-d2a17daf178f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:34.038226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830815e-c8f3-447a-9d22-d2a17daf178f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:34.041601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830815e-c8f3-447a-9d22-d2a17daf178f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:34.042039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830815e-c8f3-447a-9d22-d2a17daf178f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:34.072813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830815e-c8f3-447a-9d22-d2a17daf178f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:34.074319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830815e-c8f3-447a-9d22-d2a17daf178f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:34.091162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1063/2135] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:35.332145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857914aa-8883-4e47-b492-537304f58b83 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:35.333636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-857914aa-8883-4e47-b492-537304f58b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:35.353675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857914aa-8883-4e47-b492-537304f58b83 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:35.364900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:35.365562 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:35.365846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:35.366128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] PATH_INFO: `/v3/projects/dbe02594043f458bbe4336fb7d491b7f` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:35.371782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe02594043f458bbe4336fb7d491b7f)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:35.373950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:35.477579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe02594043f458bbe4336fb7d491b7f', 'request_id': 'req-4678e582-a613-47fe-b891-213f412ee164'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:35.490526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe02594043f458bbe4336fb7d491b7f', 'request_id': 'req-4678e582-a613-47fe-b891-213f412ee164'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:35.513700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe02594043f458bbe4336fb7d491b7f', 'request_id': 'req-4678e582-a613-47fe-b891-213f412ee164'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:35.516091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:35.550530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] The token cache is being invalidate because project dbe02594043f458bbe4336fb7d491b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:01:35.555777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:35.559548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4678e582-a613-47fe-b891-213f412ee164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:35.562559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1073/2136] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:01:35 2022] DELETE /identity/v3/projects/dbe02594043f458bbe4336fb7d491b7f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:35.567584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2a5c-7d4a-4dc1-a741-c5f8d4462d5b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:35.568089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2a5c-7d4a-4dc1-a741-c5f8d4462d5b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:35.568185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2a5c-7d4a-4dc1-a741-c5f8d4462d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:35.601173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4e6c2a5c-7d4a-4dc1-a741-c5f8d4462d5b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:35.602807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4e6c2a5c-7d4a-4dc1-a741-c5f8d4462d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:35.717963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1064/2137] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:35.725922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a9fdc-5ff0-477f-a6d5-49ccc013970e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:35.726228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a9fdc-5ff0-477f-a6d5-49ccc013970e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:35.726544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a9fdc-5ff0-477f-a6d5-49ccc013970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:35.762135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9b7a9fdc-5ff0-477f-a6d5-49ccc013970e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:35.764523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9b7a9fdc-5ff0-477f-a6d5-49ccc013970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:35.791544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1074/2138] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:35.793176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c2a5c-7d4a-4dc1-a741-c5f8d4462d5b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:35.806451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c2a5c-7d4a-4dc1-a741-c5f8d4462d5b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:35.821868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:35.822608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:35.822884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:35.823189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:35.823842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:01:35.824139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:35.828957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:35.864894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1065/2139] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:01:35.869147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a9fdc-5ff0-477f-a6d5-49ccc013970e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:35.882264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a9fdc-5ff0-477f-a6d5-49ccc013970e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:35.895943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12c6e3-8055-430b-a33f-c789ac0f17fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:35.898201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c6e3-8055-430b-a33f-c789ac0f17fb admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:35.899153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c6e3-8055-430b-a33f-c789ac0f17fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:35.899432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c6e3-8055-430b-a33f-c789ac0f17fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:35.902049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12c6e3-8055-430b-a33f-c789ac0f17fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:35.909231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12c6e3-8055-430b-a33f-c789ac0f17fb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:36.043891 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1075/2140] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:35 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:36.048118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:36.065268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3a0c9-0ce3-489c-aec7-8d78775e8439 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:36.199346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:36.263296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8287f-5503-4052-8440-0f32c923d327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:36.263939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8287f-5503-4052-8440-0f32c923d327 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.264221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8287f-5503-4052-8440-0f32c923d327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.264496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8287f-5503-4052-8440-0f32c923d327 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.265142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8287f-5503-4052-8440-0f32c923d327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:36.267119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8287f-5503-4052-8440-0f32c923d327 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:36.305035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1066/2141] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:36 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:01:36.308254 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12c6e3-8055-430b-a33f-c789ac0f17fb admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:36.325380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12c6e3-8055-430b-a33f-c789ac0f17fb admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:36.345960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9075a1-8de6-42f9-a3a4-202651946b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:36.347470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9075a1-8de6-42f9-a3a4-202651946b6f admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.348193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9075a1-8de6-42f9-a3a4-202651946b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.348892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9075a1-8de6-42f9-a3a4-202651946b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.351009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9075a1-8de6-42f9-a3a4-202651946b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:36.353892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9075a1-8de6-42f9-a3a4-202651946b6f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:36.370332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1076/2142] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:36.374265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8287f-5503-4052-8440-0f32c923d327 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:36.394987 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8287f-5503-4052-8440-0f32c923d327 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:36.428943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c1bc4-5fd9-4bae-925c-0bd2aabd0dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:36.431097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c1bc4-5fd9-4bae-925c-0bd2aabd0dfd admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.431385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c1bc4-5fd9-4bae-925c-0bd2aabd0dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.431665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c1bc4-5fd9-4bae-925c-0bd2aabd0dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.435109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c1bc4-5fd9-4bae-925c-0bd2aabd0dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:36.436779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c1bc4-5fd9-4bae-925c-0bd2aabd0dfd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:36.475919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1067/2143] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:01:36.480049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9075a1-8de6-42f9-a3a4-202651946b6f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:36.501929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9075a1-8de6-42f9-a3a4-202651946b6f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:36.517136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0150c7-8cd5-472b-9b5a-f2d73a6a133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:36.517864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0150c7-8cd5-472b-9b5a-f2d73a6a133e admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.518148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0150c7-8cd5-472b-9b5a-f2d73a6a133e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.518438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0150c7-8cd5-472b-9b5a-f2d73a6a133e admin admin] PATH_INFO: `/v3/projects/ec85bf9f75274ddd8b689fbb34f3aa43/users/7ed9f719166a44898bdffb1f07b93d67/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.538400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0150c7-8cd5-472b-9b5a-f2d73a6a133e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec85bf9f75274ddd8b689fbb34f3aa43, user_id=7ed9f719166a44898bdffb1f07b93d67, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:36.541473 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0150c7-8cd5-472b-9b5a-f2d73a6a133e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:36.568596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1077/2144] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:36 2022] PUT /identity/v3/projects/ec85bf9f75274ddd8b689fbb34f3aa43/users/7ed9f719166a44898bdffb1f07b93d67/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:36.572000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9859f2-4924-4ede-9561-13f77041e1ed None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.572723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9859f2-4924-4ede-9561-13f77041e1ed None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.573140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9859f2-4924-4ede-9561-13f77041e1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.611352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3d9859f2-4924-4ede-9561-13f77041e1ed None None] MFA Rules not processed for user `7ed9f719166a44898bdffb1f07b93d67`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:36.613302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3d9859f2-4924-4ede-9561-13f77041e1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:36.783831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0150c7-8cd5-472b-9b5a-f2d73a6a133e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:36.784595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1068/2145] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:36.901935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0150c7-8cd5-472b-9b5a-f2d73a6a133e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:36.907570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eec4f3-c3e9-4d76-99f6-5e4d0ad93f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:36.908857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eec4f3-c3e9-4d76-99f6-5e4d0ad93f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.909221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eec4f3-c3e9-4d76-99f6-5e4d0ad93f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.909501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eec4f3-c3e9-4d76-99f6-5e4d0ad93f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.910576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eec4f3-c3e9-4d76-99f6-5e4d0ad93f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:36.912148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eec4f3-c3e9-4d76-99f6-5e4d0ad93f47 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:36.950255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1078/2146] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:36 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:01:36.951435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9859f2-4924-4ede-9561-13f77041e1ed None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:36.963550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9859f2-4924-4ede-9561-13f77041e1ed None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:37.096884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853af398-7525-4e71-ab99-8fb3f0dc2776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:37.097610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853af398-7525-4e71-ab99-8fb3f0dc2776 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:37.097800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853af398-7525-4e71-ab99-8fb3f0dc2776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:37.098196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853af398-7525-4e71-ab99-8fb3f0dc2776 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:37.098995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853af398-7525-4e71-ab99-8fb3f0dc2776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:37.104549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853af398-7525-4e71-ab99-8fb3f0dc2776 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:37.165892 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eec4f3-c3e9-4d76-99f6-5e4d0ad93f47 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:37.180043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1069/2147] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:36 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:01:37.187078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eec4f3-c3e9-4d76-99f6-5e4d0ad93f47 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:37.196208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8514a-4d79-4bc4-901e-4162bee706cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:37.196976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8514a-4d79-4bc4-901e-4162bee706cb admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:37.197269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8514a-4d79-4bc4-901e-4162bee706cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:37.197538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8514a-4d79-4bc4-901e-4162bee706cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:37.198191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8514a-4d79-4bc4-901e-4162bee706cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:37.200212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8514a-4d79-4bc4-901e-4162bee706cb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:37.218501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853af398-7525-4e71-ab99-8fb3f0dc2776 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:37.224070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1079/2148] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:37.229802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853af398-7525-4e71-ab99-8fb3f0dc2776 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:37.234773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f89e37-e2e9-4609-ae5e-1a541124e814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:37.235443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f89e37-e2e9-4609-ae5e-1a541124e814 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:37.235726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f89e37-e2e9-4609-ae5e-1a541124e814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:37.236000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f89e37-e2e9-4609-ae5e-1a541124e814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:37.236648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f89e37-e2e9-4609-ae5e-1a541124e814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:37.238239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f89e37-e2e9-4609-ae5e-1a541124e814 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:37.262414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8514a-4d79-4bc4-901e-4162bee706cb admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:37.273197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8514a-4d79-4bc4-901e-4162bee706cb admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:37.274021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1070/2149] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:37.298232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde7ceb-3ae6-4dcd-9388-3ed90e7b0ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:37.299090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde7ceb-3ae6-4dcd-9388-3ed90e7b0ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:37.299446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde7ceb-3ae6-4dcd-9388-3ed90e7b0ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:37.299790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde7ceb-3ae6-4dcd-9388-3ed90e7b0ad1 admin admin] PATH_INFO: `/v3/projects/2af3d3269e5f4bb797ad7402d7cdc47f/users/a2bcef55413a4969a38cebd7561d6c84/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:37.330125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde7ceb-3ae6-4dcd-9388-3ed90e7b0ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af3d3269e5f4bb797ad7402d7cdc47f, user_id=a2bcef55413a4969a38cebd7561d6c84, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:37.332940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde7ceb-3ae6-4dcd-9388-3ed90e7b0ad1 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:37.356445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1080/2150] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:37 2022] PUT /identity/v3/projects/2af3d3269e5f4bb797ad7402d7cdc47f/users/a2bcef55413a4969a38cebd7561d6c84/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:37.388106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5e99f-f4e3-44fb-861d-aaee2c8545b5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:37.388505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5e99f-f4e3-44fb-861d-aaee2c8545b5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:37.388898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5e99f-f4e3-44fb-861d-aaee2c8545b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:37.441282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f202b0-2d20-4a1d-8467-64a6a2a8b89e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:37.441475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f202b0-2d20-4a1d-8467-64a6a2a8b89e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:37.441744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f202b0-2d20-4a1d-8467-64a6a2a8b89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:37.478123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-58a5e99f-f4e3-44fb-861d-aaee2c8545b5 None None] MFA Rules not processed for user `a2bcef55413a4969a38cebd7561d6c84`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:37.482844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-58a5e99f-f4e3-44fb-861d-aaee2c8545b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:37.505573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a0f202b0-2d20-4a1d-8467-64a6a2a8b89e None None] MFA Rules not processed for user `06d7f6b94e574f1a89c4c41b55d79044`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:37.508914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a0f202b0-2d20-4a1d-8467-64a6a2a8b89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:37.511717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:37.567468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:37.692421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1071/2151] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:37.708009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5e99f-f4e3-44fb-861d-aaee2c8545b5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:37.752813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1081/2152] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:37.771608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f202b0-2d20-4a1d-8467-64a6a2a8b89e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:37.790582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a0f202b0-2d20-4a1d-8467-64a6a2a8b89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:37.890098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5e99f-f4e3-44fb-861d-aaee2c8545b5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:37.913571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274cc8a-154b-4366-9f3b-0e7a6b180962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:37.914925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274cc8a-154b-4366-9f3b-0e7a6b180962 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:37.916311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274cc8a-154b-4366-9f3b-0e7a6b180962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:37.919320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274cc8a-154b-4366-9f3b-0e7a6b180962 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:37.923795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274cc8a-154b-4366-9f3b-0e7a6b180962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:37.931470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274cc8a-154b-4366-9f3b-0e7a6b180962 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:38.040042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1072/2153] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:37 2022] POST /identity/v3/projects => generated 375 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:38.041198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274cc8a-154b-4366-9f3b-0e7a6b180962 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:38.079342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f202b0-2d20-4a1d-8467-64a6a2a8b89e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:38.182332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274cc8a-154b-4366-9f3b-0e7a6b180962 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:38.196975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff781a8-3b53-46a9-a224-9609359b4455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:38.199001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff781a8-3b53-46a9-a224-9609359b4455 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:38.199001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff781a8-3b53-46a9-a224-9609359b4455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:38.199001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff781a8-3b53-46a9-a224-9609359b4455 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:38.199001 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff781a8-3b53-46a9-a224-9609359b4455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:38.200835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff781a8-3b53-46a9-a224-9609359b4455 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:38.263404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1073/2154] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:38 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:38.266804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff781a8-3b53-46a9-a224-9609359b4455 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:38.291719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff781a8-3b53-46a9-a224-9609359b4455 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:38.308061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:38.311241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:38.311383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc719-6cd9-4022-9990-eb5e86f1c77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:38.313401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:38.313636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc719-6cd9-4022-9990-eb5e86f1c77e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:38.313886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:38.314431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc719-6cd9-4022-9990-eb5e86f1c77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:38.315456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dc719-6cd9-4022-9990-eb5e86f1c77e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:38.317949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dc719-6cd9-4022-9990-eb5e86f1c77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:38.325247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dc719-6cd9-4022-9990-eb5e86f1c77e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:38.326010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:38.377355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1074/2155] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:38.389578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc719-6cd9-4022-9990-eb5e86f1c77e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:38.422306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dc719-6cd9-4022-9990-eb5e86f1c77e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:38.438328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf83a9-17c7-47f7-9db2-2c1982d7d273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:38.440349 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf83a9-17c7-47f7-9db2-2c1982d7d273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:38.440595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf83a9-17c7-47f7-9db2-2c1982d7d273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:38.440940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf83a9-17c7-47f7-9db2-2c1982d7d273 admin admin] PATH_INFO: `/v3/projects/233e62fd51b34997a5e095cf87f77878/users/b986d52deec24fff80fb699db3c171be/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:38.504635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf83a9-17c7-47f7-9db2-2c1982d7d273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233e62fd51b34997a5e095cf87f77878, user_id=b986d52deec24fff80fb699db3c171be, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:38.507248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf83a9-17c7-47f7-9db2-2c1982d7d273 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:38.510404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:38.530272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:38.538953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:38.543114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:38.559656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1082/2157] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 788 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:38.563123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:38.563626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1075/2157] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:38 2022] PUT /identity/v3/projects/233e62fd51b34997a5e095cf87f77878/users/b986d52deec24fff80fb699db3c171be/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:38.584763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4c62a-d75b-4c13-ac87-a6b5527bdec3 service nova] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:38.725444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09f122-73ef-4e01-bc00-fc599c759149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:38.726146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09f122-73ef-4e01-bc00-fc599c759149 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:38.726632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09f122-73ef-4e01-bc00-fc599c759149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:38.727081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09f122-73ef-4e01-bc00-fc599c759149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:38.728390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09f122-73ef-4e01-bc00-fc599c759149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:38.730094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09f122-73ef-4e01-bc00-fc599c759149 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:38.747134 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1083/2158] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:01:38.747634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf83a9-17c7-47f7-9db2-2c1982d7d273 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:38.763480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf83a9-17c7-47f7-9db2-2c1982d7d273 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:38.771884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:38.773017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:38.773586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:38.774092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] PATH_INFO: `/v3/projects/233e62fd51b34997a5e095cf87f77878/users/b986d52deec24fff80fb699db3c171be/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:38.776812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=233e62fd51b34997a5e095cf87f77878, user_id=b986d52deec24fff80fb699db3c171be, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:38.780765 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:38.808032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1076/2159] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:38 2022] PUT /identity/v3/projects/233e62fd51b34997a5e095cf87f77878/users/b986d52deec24fff80fb699db3c171be/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:38.816301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b62e3-3513-496d-8b59-fb342f7f2dfb None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:38.817178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b62e3-3513-496d-8b59-fb342f7f2dfb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:38.817456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b62e3-3513-496d-8b59-fb342f7f2dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:38.893691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-650b62e3-3513-496d-8b59-fb342f7f2dfb None None] MFA Rules not processed for user `b986d52deec24fff80fb699db3c171be`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:38.898577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-650b62e3-3513-496d-8b59-fb342f7f2dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:39.146330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1084/2160] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:39.157251 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:39.167287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:39.297542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3affc-5e95-4d9c-afca-180c9b98f96e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:39.305260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d97ec-e032-4b37-a2a9-cfbb34a82486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:39.306674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d97ec-e032-4b37-a2a9-cfbb34a82486 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:39.306674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d97ec-e032-4b37-a2a9-cfbb34a82486 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:39.306674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d97ec-e032-4b37-a2a9-cfbb34a82486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:39.428486 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d97ec-e032-4b37-a2a9-cfbb34a82486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:39.431378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d97ec-e032-4b37-a2a9-cfbb34a82486 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:39.442064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1077/2161] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:41.559783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b62e3-3513-496d-8b59-fb342f7f2dfb None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:41.583965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b62e3-3513-496d-8b59-fb342f7f2dfb None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:41.608209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:41.608856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:41.609196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:41.613140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:41.766536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:41.768330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:41.794108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1085/2162] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:42.499426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951668-f2b3-4d2c-b06d-4d0f6dc7d762 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:42.500047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951668-f2b3-4d2c-b06d-4d0f6dc7d762 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:42.500509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15951668-f2b3-4d2c-b06d-4d0f6dc7d762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:42.570741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-15951668-f2b3-4d2c-b06d-4d0f6dc7d762 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:42.581286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-15951668-f2b3-4d2c-b06d-4d0f6dc7d762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:42.849927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1078/2163] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:01:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:42.869577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:42.871624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:42.918405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:43.124971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d0cdb-32a6-49ee-a78e-cf1adc123c5e service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:43.139521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639faf-35b6-4df5-b1b5-e8b2dac6291b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:43.140648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639faf-35b6-4df5-b1b5-e8b2dac6291b service swift] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:43.142091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639faf-35b6-4df5-b1b5-e8b2dac6291b service swift] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:43.144468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639faf-35b6-4df5-b1b5-e8b2dac6291b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:43.185008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59639faf-35b6-4df5-b1b5-e8b2dac6291b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:43.192459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59639faf-35b6-4df5-b1b5-e8b2dac6291b service swift] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:43.218751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1086/2164] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:01:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:01:43.508185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15951668-f2b3-4d2c-b06d-4d0f6dc7d762 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:43.526181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15951668-f2b3-4d2c-b06d-4d0f6dc7d762 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:43.535127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe0351-2e43-445d-a94f-41f4a975e896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:43.536042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe0351-2e43-445d-a94f-41f4a975e896 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:43.536365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe0351-2e43-445d-a94f-41f4a975e896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:43.536801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe0351-2e43-445d-a94f-41f4a975e896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:43.565270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe0351-2e43-445d-a94f-41f4a975e896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:43.566735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe0351-2e43-445d-a94f-41f4a975e896 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:43.595366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1079/2165] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:48.236968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639faf-35b6-4df5-b1b5-e8b2dac6291b service swift] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:48.368319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639faf-35b6-4df5-b1b5-e8b2dac6291b service swift] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:48.382644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:48.384182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:48.385222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:48.386460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] PATH_INFO: `/v3/users/7ed9f719166a44898bdffb1f07b93d67` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:48.389184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed9f719166a44898bdffb1f07b93d67)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:48.405511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:48.507782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed9f719166a44898bdffb1f07b93d67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:48.517901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed9f719166a44898bdffb1f07b93d67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:48.518656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed9f719166a44898bdffb1f07b93d67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:48.519328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed9f719166a44898bdffb1f07b93d67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:48.532997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed9f719166a44898bdffb1f07b93d67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:48.597729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1087/2166] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:01:48 2022] DELETE /identity/v3/users/7ed9f719166a44898bdffb1f07b93d67 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:48.604043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2225b-a5db-4737-9087-9c4d6c0836dd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:48.605069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2225b-a5db-4737-9087-9c4d6c0836dd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:48.605242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2225b-a5db-4737-9087-9c4d6c0836dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:48.634606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a3f2225b-a5db-4737-9087-9c4d6c0836dd None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:48.636487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a3f2225b-a5db-4737-9087-9c4d6c0836dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:48.670577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:48.759800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1080/2167] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:48.761456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:48.762926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:48.813478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:48.993400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1127362-e779-4f2a-9706-fe2b40ba7120 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:49.002786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9bceb-149c-459c-ac05-57500f1853f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:49.005930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9bceb-149c-459c-ac05-57500f1853f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:49.006520 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9bceb-149c-459c-ac05-57500f1853f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:49.008159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9bceb-149c-459c-ac05-57500f1853f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:49.019945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9bceb-149c-459c-ac05-57500f1853f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:49.021685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9bceb-149c-459c-ac05-57500f1853f3 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:49.064476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1088/2168] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:49.498216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2225b-a5db-4737-9087-9c4d6c0836dd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:49.499779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a3f2225b-a5db-4737-9087-9c4d6c0836dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:49.531658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2225b-a5db-4737-9087-9c4d6c0836dd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:49.539299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:49.540053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:49.540415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:49.540763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] PATH_INFO: `/v3/projects/ec85bf9f75274ddd8b689fbb34f3aa43` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:49.542220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec85bf9f75274ddd8b689fbb34f3aa43)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:49.550065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:49.680861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec85bf9f75274ddd8b689fbb34f3aa43', 'request_id': 'req-761107d7-7095-4651-9c37-bf1269f8a4b6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:49.698639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec85bf9f75274ddd8b689fbb34f3aa43', 'request_id': 'req-761107d7-7095-4651-9c37-bf1269f8a4b6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:49.723582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec85bf9f75274ddd8b689fbb34f3aa43', 'request_id': 'req-761107d7-7095-4651-9c37-bf1269f8a4b6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:49.797763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] The token cache is being invalidate because project ec85bf9f75274ddd8b689fbb34f3aa43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:01:49.798064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:49.804416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:49.812280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1081/2169] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:01:49 2022] DELETE /identity/v3/projects/ec85bf9f75274ddd8b689fbb34f3aa43 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:49.812795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9bceb-149c-459c-ac05-57500f1853f3 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:49.829549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9bceb-149c-459c-ac05-57500f1853f3 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:49.926541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:49.927788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:49.928343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:49.928793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] PATH_INFO: `/v3/users/a2bcef55413a4969a38cebd7561d6c84` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:49.930708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2bcef55413a4969a38cebd7561d6c84)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:49.932860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:50.011284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2bcef55413a4969a38cebd7561d6c84'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.023090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2bcef55413a4969a38cebd7561d6c84'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.023766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2bcef55413a4969a38cebd7561d6c84'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.024735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2bcef55413a4969a38cebd7561d6c84'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.034609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2bcef55413a4969a38cebd7561d6c84'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.076163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1089/2170] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:01:49 2022] DELETE /identity/v3/users/a2bcef55413a4969a38cebd7561d6c84 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:50.544518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:50.546684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:50.563649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761107d7-7095-4651-9c37-bf1269f8a4b6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:50.650179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:50.651136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:50.651681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:50.652200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] PATH_INFO: `/v3/projects/2af3d3269e5f4bb797ad7402d7cdc47f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:50.654045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af3d3269e5f4bb797ad7402d7cdc47f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:50.657924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:50.735691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af3d3269e5f4bb797ad7402d7cdc47f', 'request_id': 'req-49c9af35-002d-4710-abfa-b2d043c817bd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.739670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:50.749085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af3d3269e5f4bb797ad7402d7cdc47f', 'request_id': 'req-49c9af35-002d-4710-abfa-b2d043c817bd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.758237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af3d3269e5f4bb797ad7402d7cdc47f', 'request_id': 'req-49c9af35-002d-4710-abfa-b2d043c817bd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.776630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] The token cache is being invalidate because project 2af3d3269e5f4bb797ad7402d7cdc47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:01:50.777175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.778074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:50.779932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1082/2171] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:01:50 2022] DELETE /identity/v3/projects/2af3d3269e5f4bb797ad7402d7cdc47f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:50.784213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:50.785778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:50.812986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a849ce-7158-484a-95f4-d78d6dfc7325 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:50.947118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:50.986082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:50.986735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:50.987055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:50.987329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] PATH_INFO: `/v3/users/b986d52deec24fff80fb699db3c171be` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:50.989109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b986d52deec24fff80fb699db3c171be)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:51.000188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:51.178296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b986d52deec24fff80fb699db3c171be'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:51.190184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b986d52deec24fff80fb699db3c171be'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:51.191132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b986d52deec24fff80fb699db3c171be'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:51.196965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b986d52deec24fff80fb699db3c171be'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:51.224868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-aae07326-7159-41f7-829a-f4dd88051385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b986d52deec24fff80fb699db3c171be'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:51.254989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1090/2172] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:01:50 2022] DELETE /identity/v3/users/b986d52deec24fff80fb699db3c171be => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:51.709082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:51.710866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:51.731492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9af35-002d-4710-abfa-b2d043c817bd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:51.865738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:51.868644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:51.869042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:51.869702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] PATH_INFO: `/v3/projects/233e62fd51b34997a5e095cf87f77878` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:51.874916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=233e62fd51b34997a5e095cf87f77878)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:51.880929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:52.003266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233e62fd51b34997a5e095cf87f77878', 'request_id': 'req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:52.014259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233e62fd51b34997a5e095cf87f77878', 'request_id': 'req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:52.024105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233e62fd51b34997a5e095cf87f77878', 'request_id': 'req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:52.046548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] The token cache is being invalidate because project 233e62fd51b34997a5e095cf87f77878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:01:52.046838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:52.047550 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1b8e066c-7eb4-45b1-8a2b-a043552f47bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:01:52.054546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dee12-a69d-4149-a20d-703d558c354a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.055455 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dee12-a69d-4149-a20d-703d558c354a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.055996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dee12-a69d-4149-a20d-703d558c354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.059871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1083/2173] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:01:51 2022] DELETE /identity/v3/projects/233e62fd51b34997a5e095cf87f77878 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:52.083124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a99dee12-a69d-4149-a20d-703d558c354a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:52.084521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a99dee12-a69d-4149-a20d-703d558c354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:52.215087 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1091/2174] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:52.223080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783a788-6de1-4bc4-bacb-7bb108c6a7f5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.223373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783a788-6de1-4bc4-bacb-7bb108c6a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.223651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783a788-6de1-4bc4-bacb-7bb108c6a7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.260189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5783a788-6de1-4bc4-bacb-7bb108c6a7f5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:52.261699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5783a788-6de1-4bc4-bacb-7bb108c6a7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:52.278761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1084/2175] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:52.283822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dee12-a69d-4149-a20d-703d558c354a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:52.312243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dee12-a69d-4149-a20d-703d558c354a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:52.321797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:52.322435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.329157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.329494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.330712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:01:52.331066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:52.333242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:52.355310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1092/2176] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:01:52.359629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783a788-6de1-4bc4-bacb-7bb108c6a7f5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:52.377447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783a788-6de1-4bc4-bacb-7bb108c6a7f5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:52.387103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafac054-2a59-49f5-8e92-5dc88186486e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:52.388020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafac054-2a59-49f5-8e92-5dc88186486e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.388538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafac054-2a59-49f5-8e92-5dc88186486e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.389420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafac054-2a59-49f5-8e92-5dc88186486e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.391371 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafac054-2a59-49f5-8e92-5dc88186486e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:52.393566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafac054-2a59-49f5-8e92-5dc88186486e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:52.451478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1085/2177] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:52 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:52.451928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:52.469554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ef357-ecd4-4338-9925-d7528fd17682 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:52.565554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:52.582928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1d832-9ed5-44a5-ac4c-14061ca64045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:52.583893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1d832-9ed5-44a5-ac4c-14061ca64045 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.584324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1d832-9ed5-44a5-ac4c-14061ca64045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.584768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1d832-9ed5-44a5-ac4c-14061ca64045 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.586979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1d832-9ed5-44a5-ac4c-14061ca64045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:52.590982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1d832-9ed5-44a5-ac4c-14061ca64045 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:52.642131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1093/2178] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:52 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:52.646166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafac054-2a59-49f5-8e92-5dc88186486e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:52.662600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafac054-2a59-49f5-8e92-5dc88186486e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:52.680352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50117a3-339e-4288-80e4-20286860fc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:52.681129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50117a3-339e-4288-80e4-20286860fc5b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.681417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50117a3-339e-4288-80e4-20286860fc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.681691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50117a3-339e-4288-80e4-20286860fc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.685517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50117a3-339e-4288-80e4-20286860fc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:52.687785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50117a3-339e-4288-80e4-20286860fc5b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:52.730521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1086/2179] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:52.732100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1d832-9ed5-44a5-ac4c-14061ca64045 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:52.752338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1d832-9ed5-44a5-ac4c-14061ca64045 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:52.767453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda7450-d2c6-4be8-a222-9ed99b3cb6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:52.768092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda7450-d2c6-4be8-a222-9ed99b3cb6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.768369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda7450-d2c6-4be8-a222-9ed99b3cb6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.768641 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda7450-d2c6-4be8-a222-9ed99b3cb6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.769296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda7450-d2c6-4be8-a222-9ed99b3cb6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:52.771799 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda7450-d2c6-4be8-a222-9ed99b3cb6a0 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:52.797387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1094/2180] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:52.798409 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50117a3-339e-4288-80e4-20286860fc5b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:52.820841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50117a3-339e-4288-80e4-20286860fc5b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:52.831506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dec21e-f30b-470b-ac1e-305aae9391d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:52.832575 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dec21e-f30b-470b-ac1e-305aae9391d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.833082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dec21e-f30b-470b-ac1e-305aae9391d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.833570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dec21e-f30b-470b-ac1e-305aae9391d3 admin admin] PATH_INFO: `/v3/projects/037f6cfeab424dd99dae2fe1347b074a/users/c939ae0713454caabd06fc894f9e58e7/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.853138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dec21e-f30b-470b-ac1e-305aae9391d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=037f6cfeab424dd99dae2fe1347b074a, user_id=c939ae0713454caabd06fc894f9e58e7, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:52.856424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dec21e-f30b-470b-ac1e-305aae9391d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:52.885740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1087/2181] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:52 2022] PUT /identity/v3/projects/037f6cfeab424dd99dae2fe1347b074a/users/c939ae0713454caabd06fc894f9e58e7/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:52.898099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e1c9-b650-4b03-b01a-fbc32f0cda82 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:52.899434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e1c9-b650-4b03-b01a-fbc32f0cda82 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:52.900757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e1c9-b650-4b03-b01a-fbc32f0cda82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:52.941165 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-7673e1c9-b650-4b03-b01a-fbc32f0cda82 None None] MFA Rules not processed for user `c939ae0713454caabd06fc894f9e58e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:52.943932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7673e1c9-b650-4b03-b01a-fbc32f0cda82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:53.058630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1095/2182] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:53.063189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dec21e-f30b-470b-ac1e-305aae9391d3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:53.087008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:53.212310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dec21e-f30b-470b-ac1e-305aae9391d3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:53.218120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92438a-1506-49f3-8721-8df8a037ab2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:53.218872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92438a-1506-49f3-8721-8df8a037ab2e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:53.219289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92438a-1506-49f3-8721-8df8a037ab2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:53.220351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92438a-1506-49f3-8721-8df8a037ab2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:53.220351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92438a-1506-49f3-8721-8df8a037ab2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:53.223199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92438a-1506-49f3-8721-8df8a037ab2e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:53.305565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1088/2183] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:53 2022] POST /identity/v3/projects => generated 383 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:53.313890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673e1c9-b650-4b03-b01a-fbc32f0cda82 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:53.336711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673e1c9-b650-4b03-b01a-fbc32f0cda82 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:53.490475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f5230-19c6-4218-ae7c-90d26fdd92a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:53.491140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f5230-19c6-4218-ae7c-90d26fdd92a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:53.491417 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f5230-19c6-4218-ae7c-90d26fdd92a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:53.491730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f5230-19c6-4218-ae7c-90d26fdd92a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:53.492368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f5230-19c6-4218-ae7c-90d26fdd92a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:53.494327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f5230-19c6-4218-ae7c-90d26fdd92a7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:53.535721 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1096/2184] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:53 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:53.539602 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92438a-1506-49f3-8721-8df8a037ab2e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:53.561793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92438a-1506-49f3-8721-8df8a037ab2e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:53.570457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a12483-edde-41c4-80dd-906ba8c21d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:53.572033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a12483-edde-41c4-80dd-906ba8c21d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:53.572644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a12483-edde-41c4-80dd-906ba8c21d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:53.573715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a12483-edde-41c4-80dd-906ba8c21d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:53.574687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a12483-edde-41c4-80dd-906ba8c21d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:53.578575 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a12483-edde-41c4-80dd-906ba8c21d91 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:53.603172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1089/2185] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:53.606179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f5230-19c6-4218-ae7c-90d26fdd92a7 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:53.618852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f5230-19c6-4218-ae7c-90d26fdd92a7 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:53.634264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf5dda-b6df-4b41-b868-c3e9e06da895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:53.634921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5dda-b6df-4b41-b868-c3e9e06da895 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:53.636477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5dda-b6df-4b41-b868-c3e9e06da895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:53.636769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5dda-b6df-4b41-b868-c3e9e06da895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:53.637471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf5dda-b6df-4b41-b868-c3e9e06da895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:53.639106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf5dda-b6df-4b41-b868-c3e9e06da895 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:53.666963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1097/2186] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:53.667939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a12483-edde-41c4-80dd-906ba8c21d91 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:53.685589 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a12483-edde-41c4-80dd-906ba8c21d91 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:53.702415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00bfd-e239-47d4-a2e6-0dec0b512250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:53.703723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00bfd-e239-47d4-a2e6-0dec0b512250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:53.704234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00bfd-e239-47d4-a2e6-0dec0b512250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:53.704518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00bfd-e239-47d4-a2e6-0dec0b512250 admin admin] PATH_INFO: `/v3/projects/780ecb86f7654fb58d8507ebcc446aa4/users/de8bcb70368748eb882311119a557c74/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:53.761161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d00bfd-e239-47d4-a2e6-0dec0b512250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780ecb86f7654fb58d8507ebcc446aa4, user_id=de8bcb70368748eb882311119a557c74, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:53.769883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d00bfd-e239-47d4-a2e6-0dec0b512250 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:53.809543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1090/2187] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:53 2022] PUT /identity/v3/projects/780ecb86f7654fb58d8507ebcc446aa4/users/de8bcb70368748eb882311119a557c74/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:53.818572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffb6ec-0f22-477d-8465-141a2a57c663 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:53.818865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffb6ec-0f22-477d-8465-141a2a57c663 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:53.819167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffb6ec-0f22-477d-8465-141a2a57c663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:53.892124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bdffb6ec-0f22-477d-8465-141a2a57c663 None None] MFA Rules not processed for user `de8bcb70368748eb882311119a557c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:53.898106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bdffb6ec-0f22-477d-8465-141a2a57c663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:53.989521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:54.083380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1098/2188] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:54.085838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00bfd-e239-47d4-a2e6-0dec0b512250 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:54.182553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00bfd-e239-47d4-a2e6-0dec0b512250 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:54.193047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15032a6-5a2e-4207-b470-300c4e366fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:54.193795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15032a6-5a2e-4207-b470-300c4e366fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:54.195304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15032a6-5a2e-4207-b470-300c4e366fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:54.195751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15032a6-5a2e-4207-b470-300c4e366fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:54.196440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15032a6-5a2e-4207-b470-300c4e366fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:54.200386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15032a6-5a2e-4207-b470-300c4e366fe7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:54.247504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1091/2189] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:01:54 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:54.251305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffb6ec-0f22-477d-8465-141a2a57c663 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:54.268443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffb6ec-0f22-477d-8465-141a2a57c663 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:54.378774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e736-7271-4e16-a137-7e390cf9ec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:54.379469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e736-7271-4e16-a137-7e390cf9ec79 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:54.379746 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e736-7271-4e16-a137-7e390cf9ec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:54.380022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e736-7271-4e16-a137-7e390cf9ec79 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:54.380660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9e736-7271-4e16-a137-7e390cf9ec79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:54.382678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9e736-7271-4e16-a137-7e390cf9ec79 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:54.427055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1099/2190] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:01:54 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:01:54.431499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15032a6-5a2e-4207-b470-300c4e366fe7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:54.455327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15032a6-5a2e-4207-b470-300c4e366fe7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:54.474958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d7064-14b7-4af9-91d2-f3cd9a6b16bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:54.474958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d7064-14b7-4af9-91d2-f3cd9a6b16bc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:54.474958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d7064-14b7-4af9-91d2-f3cd9a6b16bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:54.474958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d7064-14b7-4af9-91d2-f3cd9a6b16bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:54.475415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d7064-14b7-4af9-91d2-f3cd9a6b16bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:54.476595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d7064-14b7-4af9-91d2-f3cd9a6b16bc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:54.489422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1092/2191] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:54.492829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e736-7271-4e16-a137-7e390cf9ec79 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:54.507770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e736-7271-4e16-a137-7e390cf9ec79 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:54.512982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aaf11-5128-4eb1-a088-cc40d5fe6b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:54.513928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aaf11-5128-4eb1-a088-cc40d5fe6b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:54.514497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aaf11-5128-4eb1-a088-cc40d5fe6b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:54.514876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aaf11-5128-4eb1-a088-cc40d5fe6b03 admin admin] PATH_INFO: `/v3/projects/212fde1e79f74dd99efeda32b17ae110/users/fe132be120a14c4b832b3c1c18cb786e/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:54.534024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8aaf11-5128-4eb1-a088-cc40d5fe6b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212fde1e79f74dd99efeda32b17ae110, user_id=fe132be120a14c4b832b3c1c18cb786e, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:54.537549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8aaf11-5128-4eb1-a088-cc40d5fe6b03 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:54.561779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1100/2192] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:54 2022] PUT /identity/v3/projects/212fde1e79f74dd99efeda32b17ae110/users/fe132be120a14c4b832b3c1c18cb786e/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:54.565848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d7064-14b7-4af9-91d2-f3cd9a6b16bc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:54.590393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d7064-14b7-4af9-91d2-f3cd9a6b16bc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:54.632673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:54.723919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911403b5-e6c3-4c27-b657-e8ee679451dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:54.724605 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911403b5-e6c3-4c27-b657-e8ee679451dc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:54.725142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911403b5-e6c3-4c27-b657-e8ee679451dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:54.725717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911403b5-e6c3-4c27-b657-e8ee679451dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:54.726755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911403b5-e6c3-4c27-b657-e8ee679451dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:54.730159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911403b5-e6c3-4c27-b657-e8ee679451dc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:54.746194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1093/2193] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:01:54.750357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aaf11-5128-4eb1-a088-cc40d5fe6b03 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:54.767467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aaf11-5128-4eb1-a088-cc40d5fe6b03 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:54.781195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:54.782884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:54.784427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:54.784882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] PATH_INFO: `/v3/projects/212fde1e79f74dd99efeda32b17ae110/users/fe132be120a14c4b832b3c1c18cb786e/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:54.791538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212fde1e79f74dd99efeda32b17ae110, user_id=fe132be120a14c4b832b3c1c18cb786e, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:54.798436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:54.833926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1101/2194] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:01:54 2022] PUT /identity/v3/projects/212fde1e79f74dd99efeda32b17ae110/users/fe132be120a14c4b832b3c1c18cb786e/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:01:54.845278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f927a1-7960-4053-9052-1cff3c04e780 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:54.845656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f927a1-7960-4053-9052-1cff3c04e780 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:54.846024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f927a1-7960-4053-9052-1cff3c04e780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:54.917040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c9f927a1-7960-4053-9052-1cff3c04e780 None None] MFA Rules not processed for user `fe132be120a14c4b832b3c1c18cb786e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:54.917752 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c9f927a1-7960-4053-9052-1cff3c04e780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:55.093043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1094/2195] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:55.106099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:55.108795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:55.250226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3a607-032b-4f17-8d34-6e3492812d43 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:55.256803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24291a70-fb16-4914-aedc-db4aec48fc83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:55.259297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24291a70-fb16-4914-aedc-db4aec48fc83 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:55.260776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24291a70-fb16-4914-aedc-db4aec48fc83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:55.260916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24291a70-fb16-4914-aedc-db4aec48fc83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:55.371180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:55.394542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24291a70-fb16-4914-aedc-db4aec48fc83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:55.397364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24291a70-fb16-4914-aedc-db4aec48fc83 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:55.411661 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1102/2196] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:57.262552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f927a1-7960-4053-9052-1cff3c04e780 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:57.264038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c9f927a1-7960-4053-9052-1cff3c04e780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:57.285189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f927a1-7960-4053-9052-1cff3c04e780 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:57.294318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ac675-962d-4e4c-b720-b1ddd2226a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:57.296140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ac675-962d-4e4c-b720-b1ddd2226a0a service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:57.296140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ac675-962d-4e4c-b720-b1ddd2226a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:57.296140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ac675-962d-4e4c-b720-b1ddd2226a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:57.428005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ac675-962d-4e4c-b720-b1ddd2226a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:57.429991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ac675-962d-4e4c-b720-b1ddd2226a0a service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:57.441180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1095/2197] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:58.312690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24291a70-fb16-4914-aedc-db4aec48fc83 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:58.321828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24291a70-fb16-4914-aedc-db4aec48fc83 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:58.325474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776e4d4-9101-42cc-b402-fedc48c602f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:58.326072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776e4d4-9101-42cc-b402-fedc48c602f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:58.326348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776e4d4-9101-42cc-b402-fedc48c602f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:58.326617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776e4d4-9101-42cc-b402-fedc48c602f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:58.336612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776e4d4-9101-42cc-b402-fedc48c602f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:58.338141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776e4d4-9101-42cc-b402-fedc48c602f5 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:58.348818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1103/2198] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:58.391877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb11d32-70cb-482b-8f50-9d1360dfcaec None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:58.392400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb11d32-70cb-482b-8f50-9d1360dfcaec None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:58.392511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb11d32-70cb-482b-8f50-9d1360dfcaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:58.415006 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1cb11d32-70cb-482b-8f50-9d1360dfcaec None None] MFA Rules not processed for user `99b3e727e6fa480c90a054a60b68b8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:58.416230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1cb11d32-70cb-482b-8f50-9d1360dfcaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:58.464098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:58.475208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1096/2199] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:01:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:02:02.253695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776e4d4-9101-42cc-b402-fedc48c602f5 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:02.313551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776e4d4-9101-42cc-b402-fedc48c602f5 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:02.318042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:02.318658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:02.318964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:02.319232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] PATH_INFO: `/v3/users/c939ae0713454caabd06fc894f9e58e7` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:02.320364 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c939ae0713454caabd06fc894f9e58e7)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:02.322248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:02.365258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c939ae0713454caabd06fc894f9e58e7'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:02.372586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c939ae0713454caabd06fc894f9e58e7'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:02.373237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c939ae0713454caabd06fc894f9e58e7'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:02.373829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c939ae0713454caabd06fc894f9e58e7'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:02.380512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c939ae0713454caabd06fc894f9e58e7'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:02.397910 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1104/2200] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:02:02 2022] DELETE /identity/v3/users/c939ae0713454caabd06fc894f9e58e7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:02.402534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00426-bd0b-41a3-839b-4826cc95a0a6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:02.403023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00426-bd0b-41a3-839b-4826cc95a0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:02.403140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00426-bd0b-41a3-839b-4826cc95a0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:02.423250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b5b00426-bd0b-41a3-839b-4826cc95a0a6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:02.424454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b5b00426-bd0b-41a3-839b-4826cc95a0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:02.485277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1097/2201] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:02:02.495610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:02.497096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:02.548459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:02.565017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bb158-38ae-44b4-b97a-4cefddfd158e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:02.568481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17add74-9860-4219-a444-b2fc3e8b8ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:02.569250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17add74-9860-4219-a444-b2fc3e8b8ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:02.569528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17add74-9860-4219-a444-b2fc3e8b8ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:02.569899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17add74-9860-4219-a444-b2fc3e8b8ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:02.579868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17add74-9860-4219-a444-b2fc3e8b8ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:02.581424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17add74-9860-4219-a444-b2fc3e8b8ce6 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:02.592406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1105/2202] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:02.949387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b00426-bd0b-41a3-839b-4826cc95a0a6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:02.951364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b5b00426-bd0b-41a3-839b-4826cc95a0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:02.971033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b00426-bd0b-41a3-839b-4826cc95a0a6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:02.982175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:02.983331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:02.983797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:02.984258 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] PATH_INFO: `/v3/projects/037f6cfeab424dd99dae2fe1347b074a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:02.985583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] RBAC: Authorizing `identity:delete_project(project_id=037f6cfeab424dd99dae2fe1347b074a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:02.988504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:03.051984 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '037f6cfeab424dd99dae2fe1347b074a', 'request_id': 'req-8d6196e8-19e4-44f6-a1a4-d3268366581d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.060910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '037f6cfeab424dd99dae2fe1347b074a', 'request_id': 'req-8d6196e8-19e4-44f6-a1a4-d3268366581d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.067979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '037f6cfeab424dd99dae2fe1347b074a', 'request_id': 'req-8d6196e8-19e4-44f6-a1a4-d3268366581d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.080421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] The token cache is being invalidate because project 037f6cfeab424dd99dae2fe1347b074a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:02:03.080713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.081365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8d6196e8-19e4-44f6-a1a4-d3268366581d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.083668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1098/2203] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:02:02 2022] DELETE /identity/v3/projects/037f6cfeab424dd99dae2fe1347b074a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:03.088216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17add74-9860-4219-a444-b2fc3e8b8ce6 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:03.097708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17add74-9860-4219-a444-b2fc3e8b8ce6 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:03.169154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:03.169939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:03.170328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:03.170803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] PATH_INFO: `/v3/users/de8bcb70368748eb882311119a557c74` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:03.172108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8bcb70368748eb882311119a557c74)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:03.175795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:03.241404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8bcb70368748eb882311119a557c74'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.256062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8bcb70368748eb882311119a557c74'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.258930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8bcb70368748eb882311119a557c74'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.262273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8bcb70368748eb882311119a557c74'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.277246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8bcb70368748eb882311119a557c74'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:03.306213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1106/2204] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:02:03 2022] DELETE /identity/v3/users/de8bcb70368748eb882311119a557c74 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:03.722604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8519-9bdc-4d56-8d89-32d59470027f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:03.723621 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8519-9bdc-4d56-8d89-32d59470027f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:03.724326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8519-9bdc-4d56-8d89-32d59470027f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:03.730832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:03.732299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:03.757608 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-625f8519-9bdc-4d56-8d89-32d59470027f None None] MFA Rules not processed for user `ce6a1b5ca7584f90bfe83b73aff1335a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:03.761763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-625f8519-9bdc-4d56-8d89-32d59470027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:03.866187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1099/2205] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:02:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:03.883560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3709-bdb5-4b58-ac2d-eb1aaeca6b45 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:03.913904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:03.914946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:03.915637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:03.915637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] PATH_INFO: `/v3/projects/780ecb86f7654fb58d8507ebcc446aa4` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:03.916853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780ecb86f7654fb58d8507ebcc446aa4)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:03.918840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:03.930245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:04.002074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780ecb86f7654fb58d8507ebcc446aa4', 'request_id': 'req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.022199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780ecb86f7654fb58d8507ebcc446aa4', 'request_id': 'req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.040465 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780ecb86f7654fb58d8507ebcc446aa4', 'request_id': 'req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.060391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd839ce-077a-4a43-8485-964885a887d6 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.060687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd839ce-077a-4a43-8485-964885a887d6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.060981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd839ce-077a-4a43-8485-964885a887d6 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.062407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1100/2206] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 12:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:02:04.065525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] The token cache is being invalidate because project 780ecb86f7654fb58d8507ebcc446aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:02:04.065996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.067102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.068821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1107/2207] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:02:03 2022] DELETE /identity/v3/projects/780ecb86f7654fb58d8507ebcc446aa4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:04.071492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc912f6-3931-401e-a23b-70a98a38d808 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.071950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc912f6-3931-401e-a23b-70a98a38d808 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.072329 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc912f6-3931-401e-a23b-70a98a38d808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.076306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:04.083660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.097531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6cc912f6-3931-401e-a23b-70a98a38d808 None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:04.104597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6cc912f6-3931-401e-a23b-70a98a38d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.136568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:04.186711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2d844-21c2-4fd8-8c9d-eb2b3618d3b8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:04.195655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:04.196628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.197309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.197660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] PATH_INFO: `/v3/users/fe132be120a14c4b832b3c1c18cb786e` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.199243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe132be120a14c4b832b3c1c18cb786e)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:04.202913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:04.234167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1101/2208] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 12:02:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:04.244479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc912f6-3931-401e-a23b-70a98a38d808 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:04.246419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6cc912f6-3931-401e-a23b-70a98a38d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.271872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe132be120a14c4b832b3c1c18cb786e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.280435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe132be120a14c4b832b3c1c18cb786e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.281153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe132be120a14c4b832b3c1c18cb786e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.281970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe132be120a14c4b832b3c1c18cb786e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.292415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ef3d8a89-c689-458b-8ed9-4f71f9e90552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe132be120a14c4b832b3c1c18cb786e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.312884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc912f6-3931-401e-a23b-70a98a38d808 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:04.317128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900bf7d-3098-4fe3-ac24-39a75a4a620f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6774fdd8b2e46b6a7d00919568f9fd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:04.317736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900bf7d-3098-4fe3-ac24-39a75a4a620f service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.318278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900bf7d-3098-4fe3-ac24-39a75a4a620f service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.318390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1108/2209] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:02:04 2022] DELETE /identity/v3/users/fe132be120a14c4b832b3c1c18cb786e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:04.318509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900bf7d-3098-4fe3-ac24-39a75a4a620f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.320688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0900bf7d-3098-4fe3-ac24-39a75a4a620f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.337404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900bf7d-3098-4fe3-ac24-39a75a4a620f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:04.338944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900bf7d-3098-4fe3-ac24-39a75a4a620f service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:04.353543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1102/2210] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:02:04.461045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae2f20-35e3-4c4f-b188-4b98996bbffa None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.461673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae2f20-35e3-4c4f-b188-4b98996bbffa None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.461992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae2f20-35e3-4c4f-b188-4b98996bbffa None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.464198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1109/2211] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 12:02:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:04.472544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d0d5f-1970-4170-b5f0-7d49fd386d30 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.472904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d0d5f-1970-4170-b5f0-7d49fd386d30 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.473259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d0d5f-1970-4170-b5f0-7d49fd386d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.498687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-081d0d5f-1970-4170-b5f0-7d49fd386d30 None None] MFA Rules not processed for user `48ad70aa047d443b97e13447397a8a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:04.499963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-081d0d5f-1970-4170-b5f0-7d49fd386d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.568911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1103/2212] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 12:02:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:04.594720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae2f20-35e3-4c4f-b188-4b98996bbffa None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:04.596692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ffae2f20-35e3-4c4f-b188-4b98996bbffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.612092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae2f20-35e3-4c4f-b188-4b98996bbffa None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:04.615633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8d53b-8f24-4ed6-9c89-f17aba288184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:04.616334 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d53b-8f24-4ed6-9c89-f17aba288184 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.616823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d53b-8f24-4ed6-9c89-f17aba288184 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.617221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8d53b-8f24-4ed6-9c89-f17aba288184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.625342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8d53b-8f24-4ed6-9c89-f17aba288184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:04.627126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8d53b-8f24-4ed6-9c89-f17aba288184 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:04.633315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:04.638117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1110/2213] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:04.696946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d0d5f-1970-4170-b5f0-7d49fd386d30 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:04.699017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-081d0d5f-1970-4170-b5f0-7d49fd386d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.770871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d0d5f-1970-4170-b5f0-7d49fd386d30 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:04.776028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:04.776689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.776969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.777290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] PATH_INFO: `/v3/projects/212fde1e79f74dd99efeda32b17ae110` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.778348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=212fde1e79f74dd99efeda32b17ae110)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:04.780345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:04.795884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:04.839842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '212fde1e79f74dd99efeda32b17ae110', 'request_id': 'req-53e4a946-d0a3-4a6b-bbac-e26be4306f09'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.849988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '212fde1e79f74dd99efeda32b17ae110', 'request_id': 'req-53e4a946-d0a3-4a6b-bbac-e26be4306f09'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.850216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d33b-619d-41db-88f2-95dd3423da00 None None] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.850216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d33b-619d-41db-88f2-95dd3423da00 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.850440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d33b-619d-41db-88f2-95dd3423da00 None None] PATH_INFO: `/` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.852955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1111/2214] 10.222.0.52 () {54 vars in 921 bytes} [Mon Oct 10 12:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:04.858571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe23ef-d650-4f60-8f08-2bb9b0012a1e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.858868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe23ef-d650-4f60-8f08-2bb9b0012a1e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.859174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe23ef-d650-4f60-8f08-2bb9b0012a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.859588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '212fde1e79f74dd99efeda32b17ae110', 'request_id': 'req-53e4a946-d0a3-4a6b-bbac-e26be4306f09'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.872899 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] The token cache is being invalidate because project 212fde1e79f74dd99efeda32b17ae110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:02:04.873199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.873879 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-53e4a946-d0a3-4a6b-bbac-e26be4306f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:04.879312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1104/2215] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:02:04 2022] DELETE /identity/v3/projects/212fde1e79f74dd99efeda32b17ae110 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:04.880956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f450-9b27-42a5-91a0-e139c377a570 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:04.881235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f450-9b27-42a5-91a0-e139c377a570 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:04.881357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bebe23ef-d650-4f60-8f08-2bb9b0012a1e None None] MFA Rules not processed for user `152edb19fd7b434485b0e1ebc5078335`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:04.881529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f450-9b27-42a5-91a0-e139c377a570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:04.882489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bebe23ef-d650-4f60-8f08-2bb9b0012a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.904397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5942f450-9b27-42a5-91a0-e139c377a570 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:04.905291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5942f450-9b27-42a5-91a0-e139c377a570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.972547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1112/2216] 10.222.0.52 () {58 vars in 1018 bytes} [Mon Oct 10 12:02:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:02:04.983923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe23ef-d650-4f60-8f08-2bb9b0012a1e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:04.985293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bebe23ef-d650-4f60-8f08-2bb9b0012a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:04.994514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1105/2217] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:05.004565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd874a-c5ff-4313-8076-a07b3109fb75 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.004948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd874a-c5ff-4313-8076-a07b3109fb75 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.005288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd874a-c5ff-4313-8076-a07b3109fb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.026487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-07bd874a-c5ff-4313-8076-a07b3109fb75 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:05.028120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-07bd874a-c5ff-4313-8076-a07b3109fb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:05.037875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1106/2218] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:05.042430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd874a-c5ff-4313-8076-a07b3109fb75 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:05.055451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd874a-c5ff-4313-8076-a07b3109fb75 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.060298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe23ef-d650-4f60-8f08-2bb9b0012a1e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.060458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.060968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.061169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.061557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.062559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:02:05.062874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.064797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5f12f-63cd-4af5-9af8-09013149eee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.065557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:05.065686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5f12f-63cd-4af5-9af8-09013149eee9 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.065940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5f12f-63cd-4af5-9af8-09013149eee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.066306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5f12f-63cd-4af5-9af8-09013149eee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.078194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5f12f-63cd-4af5-9af8-09013149eee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.079839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5f12f-63cd-4af5-9af8-09013149eee9 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:05.086678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1107/2219] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:02:05.091572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:05.097429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1113/2220] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:02:05.108554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15028b0-a4f7-4950-b014-c4573f50949c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.115900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3ca69-8ad4-4dc4-b3dc-be97d7466814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.117214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3ca69-8ad4-4dc4-b3dc-be97d7466814 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.117573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3ca69-8ad4-4dc4-b3dc-be97d7466814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.117920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3ca69-8ad4-4dc4-b3dc-be97d7466814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.118650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3ca69-8ad4-4dc4-b3dc-be97d7466814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.120809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3ca69-8ad4-4dc4-b3dc-be97d7466814 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:05.155134 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1108/2221] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:02:05 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:02:05.158200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5f12f-63cd-4af5-9af8-09013149eee9 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:05.170685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5f12f-63cd-4af5-9af8-09013149eee9 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.251514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1312a5-9257-4e22-a74f-be732bcaf235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.252142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1312a5-9257-4e22-a74f-be732bcaf235 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.252464 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1312a5-9257-4e22-a74f-be732bcaf235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.252740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1312a5-9257-4e22-a74f-be732bcaf235 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.253826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1312a5-9257-4e22-a74f-be732bcaf235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.256277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1312a5-9257-4e22-a74f-be732bcaf235 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:05.292301 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1114/2222] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:02:05 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:02:05.295926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3ca69-8ad4-4dc4-b3dc-be97d7466814 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:05.310030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3ca69-8ad4-4dc4-b3dc-be97d7466814 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.315455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c92a7-2098-4c6b-8dc9-661ff6d71a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.316064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c92a7-2098-4c6b-8dc9-661ff6d71a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.316351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c92a7-2098-4c6b-8dc9-661ff6d71a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.316628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c92a7-2098-4c6b-8dc9-661ff6d71a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.317335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c92a7-2098-4c6b-8dc9-661ff6d71a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.319112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c92a7-2098-4c6b-8dc9-661ff6d71a13 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:05.332869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1109/2223] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:02:05.334039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1312a5-9257-4e22-a74f-be732bcaf235 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:05.355883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1312a5-9257-4e22-a74f-be732bcaf235 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.361565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7024e-002d-498e-8e97-6d40ce4c8ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.362290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7024e-002d-498e-8e97-6d40ce4c8ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.362670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7024e-002d-498e-8e97-6d40ce4c8ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.363180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7024e-002d-498e-8e97-6d40ce4c8ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.363921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7024e-002d-498e-8e97-6d40ce4c8ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.365605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7024e-002d-498e-8e97-6d40ce4c8ef8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:05.401015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1115/2224] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:02:05.405326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c92a7-2098-4c6b-8dc9-661ff6d71a13 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:05.436156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c92a7-2098-4c6b-8dc9-661ff6d71a13 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.447324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e301d0-a412-4183-9b73-f3c2ff181f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.448027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e301d0-a412-4183-9b73-f3c2ff181f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.448307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e301d0-a412-4183-9b73-f3c2ff181f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.448587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e301d0-a412-4183-9b73-f3c2ff181f0b admin admin] PATH_INFO: `/v3/projects/4f7d52a627fc46d58caf9161add059cb/users/df2a80432f58487c87bba8dff5fe75fb/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.479980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e301d0-a412-4183-9b73-f3c2ff181f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f7d52a627fc46d58caf9161add059cb, user_id=df2a80432f58487c87bba8dff5fe75fb, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.483249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e301d0-a412-4183-9b73-f3c2ff181f0b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:05.534192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1110/2225] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:02:05 2022] PUT /identity/v3/projects/4f7d52a627fc46d58caf9161add059cb/users/df2a80432f58487c87bba8dff5fe75fb/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:05.536556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750a704-dac3-4d18-99eb-cd4aeda15c4f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.536989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750a704-dac3-4d18-99eb-cd4aeda15c4f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.537575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750a704-dac3-4d18-99eb-cd4aeda15c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.572867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:05.580116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-b750a704-dac3-4d18-99eb-cd4aeda15c4f None None] MFA Rules not processed for user `df2a80432f58487c87bba8dff5fe75fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:05.582042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b750a704-dac3-4d18-99eb-cd4aeda15c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:05.695268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1116/2226] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:02:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:05.699772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e301d0-a412-4183-9b73-f3c2ff181f0b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:05.733114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:05.803308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e301d0-a412-4183-9b73-f3c2ff181f0b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.812440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a59332-dc80-4d9b-9181-5a6888a87ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.813315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a59332-dc80-4d9b-9181-5a6888a87ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.813739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a59332-dc80-4d9b-9181-5a6888a87ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.814177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a59332-dc80-4d9b-9181-5a6888a87ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.815033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a59332-dc80-4d9b-9181-5a6888a87ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.817440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a59332-dc80-4d9b-9181-5a6888a87ed0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:05.860247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1111/2227] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:02:05 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:02:05.861341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750a704-dac3-4d18-99eb-cd4aeda15c4f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:05.875921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750a704-dac3-4d18-99eb-cd4aeda15c4f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:05.956774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0f608-600c-4809-9f82-63f743427511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:05.957550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0f608-600c-4809-9f82-63f743427511 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:05.957935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0f608-600c-4809-9f82-63f743427511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:05.958315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0f608-600c-4809-9f82-63f743427511 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:05.959130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb0f608-600c-4809-9f82-63f743427511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:05.961576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb0f608-600c-4809-9f82-63f743427511 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.007563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1117/2228] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:02:05 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:02:06.008868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a59332-dc80-4d9b-9181-5a6888a87ed0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.022728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a59332-dc80-4d9b-9181-5a6888a87ed0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.029917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e04a-ffbc-4090-9108-03ddbba1884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.030782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e04a-ffbc-4090-9108-03ddbba1884d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.031433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e04a-ffbc-4090-9108-03ddbba1884d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.032000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e04a-ffbc-4090-9108-03ddbba1884d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.033195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e04a-ffbc-4090-9108-03ddbba1884d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.035388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e04a-ffbc-4090-9108-03ddbba1884d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.051594 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1112/2229] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:02:06.056346 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0f608-600c-4809-9f82-63f743427511 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.068679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0f608-600c-4809-9f82-63f743427511 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.073968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e518707-168e-4b0f-88a7-229db9a0fc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.075559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e518707-168e-4b0f-88a7-229db9a0fc2e admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.076093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e518707-168e-4b0f-88a7-229db9a0fc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.076512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e518707-168e-4b0f-88a7-229db9a0fc2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.077450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e518707-168e-4b0f-88a7-229db9a0fc2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.079347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e518707-168e-4b0f-88a7-229db9a0fc2e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.092138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1118/2230] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:02:06.096512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e04a-ffbc-4090-9108-03ddbba1884d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.107293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e04a-ffbc-4090-9108-03ddbba1884d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.113453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273fb97-fbc7-4021-8f47-0238a585e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.114108 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273fb97-fbc7-4021-8f47-0238a585e5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.114393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273fb97-fbc7-4021-8f47-0238a585e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.114695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273fb97-fbc7-4021-8f47-0238a585e5f1 admin admin] PATH_INFO: `/v3/projects/f3d08ff030ad4fbf988bf4cdfafc4473/users/a7a37fd440ba4ec2b91edb2acc3898af/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.143477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273fb97-fbc7-4021-8f47-0238a585e5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d08ff030ad4fbf988bf4cdfafc4473, user_id=a7a37fd440ba4ec2b91edb2acc3898af, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.146298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273fb97-fbc7-4021-8f47-0238a585e5f1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.166387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1113/2231] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:02:06 2022] PUT /identity/v3/projects/f3d08ff030ad4fbf988bf4cdfafc4473/users/a7a37fd440ba4ec2b91edb2acc3898af/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:06.178411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0561d-0d7f-49cf-b61a-f361f32ce17c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.178747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0561d-0d7f-49cf-b61a-f361f32ce17c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.179648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0561d-0d7f-49cf-b61a-f361f32ce17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.208000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-10f0561d-0d7f-49cf-b61a-f361f32ce17c None None] MFA Rules not processed for user `a7a37fd440ba4ec2b91edb2acc3898af`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:06.209240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-10f0561d-0d7f-49cf-b61a-f361f32ce17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:06.297191 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1119/2232] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:02:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:06.298849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273fb97-fbc7-4021-8f47-0238a585e5f1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.366408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273fb97-fbc7-4021-8f47-0238a585e5f1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.371754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13de3d6-e439-4050-858e-657f585a9ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.373432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13de3d6-e439-4050-858e-657f585a9ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.373432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13de3d6-e439-4050-858e-657f585a9ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.373695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13de3d6-e439-4050-858e-657f585a9ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.374528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13de3d6-e439-4050-858e-657f585a9ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.376621 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13de3d6-e439-4050-858e-657f585a9ec3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.406428 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1114/2233] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:02:06 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:02:06.407588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0561d-0d7f-49cf-b61a-f361f32ce17c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.419856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0561d-0d7f-49cf-b61a-f361f32ce17c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.429333 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:06.481237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8ee66-e5cf-4703-8951-76976196b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.482235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8ee66-e5cf-4703-8951-76976196b351 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.482235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8ee66-e5cf-4703-8951-76976196b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.482442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8ee66-e5cf-4703-8951-76976196b351 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.483067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8ee66-e5cf-4703-8951-76976196b351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.488816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8ee66-e5cf-4703-8951-76976196b351 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.512850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1120/2234] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:02:06 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:02:06.517116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13de3d6-e439-4050-858e-657f585a9ec3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.528622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13de3d6-e439-4050-858e-657f585a9ec3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.534237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33efb66-6fee-45c1-ab7a-057466b911e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.534845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33efb66-6fee-45c1-ab7a-057466b911e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.535153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33efb66-6fee-45c1-ab7a-057466b911e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.535438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33efb66-6fee-45c1-ab7a-057466b911e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.536080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33efb66-6fee-45c1-ab7a-057466b911e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.537671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33efb66-6fee-45c1-ab7a-057466b911e1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.549277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1115/2235] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:02:06.554201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8ee66-e5cf-4703-8951-76976196b351 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.564676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8ee66-e5cf-4703-8951-76976196b351 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.569332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0565-a47d-4340-9202-790ea82f4642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.570492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0565-a47d-4340-9202-790ea82f4642 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.570794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0565-a47d-4340-9202-790ea82f4642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.571114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0565-a47d-4340-9202-790ea82f4642 admin admin] PATH_INFO: `/v3/projects/c020fa0bee044e349dff4fbb14b1f9e9/users/070aab1569d643a8a7983d601c3eab06/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.586284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df0565-a47d-4340-9202-790ea82f4642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c020fa0bee044e349dff4fbb14b1f9e9, user_id=070aab1569d643a8a7983d601c3eab06, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.588965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df0565-a47d-4340-9202-790ea82f4642 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.602106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1121/2236] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:02:06 2022] PUT /identity/v3/projects/c020fa0bee044e349dff4fbb14b1f9e9/users/070aab1569d643a8a7983d601c3eab06/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:06.606075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33efb66-6fee-45c1-ab7a-057466b911e1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.615961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33efb66-6fee-45c1-ab7a-057466b911e1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.660600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:06.673998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfb48f-4947-432d-8397-62e540d5923b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.674591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfb48f-4947-432d-8397-62e540d5923b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.674871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfb48f-4947-432d-8397-62e540d5923b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.675165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfb48f-4947-432d-8397-62e540d5923b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.675788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfb48f-4947-432d-8397-62e540d5923b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.677650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfb48f-4947-432d-8397-62e540d5923b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.687197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1116/2237] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:02:06.691321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0565-a47d-4340-9202-790ea82f4642 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.702006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0565-a47d-4340-9202-790ea82f4642 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.706525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.707206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.707488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.707759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] PATH_INFO: `/v3/projects/c020fa0bee044e349dff4fbb14b1f9e9/users/070aab1569d643a8a7983d601c3eab06/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.709671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c020fa0bee044e349dff4fbb14b1f9e9, user_id=070aab1569d643a8a7983d601c3eab06, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.712851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.723127 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1122/2238] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:02:06 2022] PUT /identity/v3/projects/c020fa0bee044e349dff4fbb14b1f9e9/users/070aab1569d643a8a7983d601c3eab06/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:02:06.733505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe7a7a-373b-4949-82d6-1e36bd1410f1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.733798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe7a7a-373b-4949-82d6-1e36bd1410f1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.734070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe7a7a-373b-4949-82d6-1e36bd1410f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.762817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c5fe7a7a-373b-4949-82d6-1e36bd1410f1 None None] MFA Rules not processed for user `070aab1569d643a8a7983d601c3eab06`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:06.764065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c5fe7a7a-373b-4949-82d6-1e36bd1410f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:06.831421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1117/2239] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:02:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:06.843358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:06.845026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:06.901897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216671-84c2-47f5-a779-d69426c5766e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:06.906299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:06.906938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:06.907225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:06.907498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:06.967041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:06.968523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:06.977814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1123/2240] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:08.496479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe7a7a-373b-4949-82d6-1e36bd1410f1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:08.508598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe7a7a-373b-4949-82d6-1e36bd1410f1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:08.512348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45eef2-c2b7-484d-9923-e43ee09fe454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:08.513028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45eef2-c2b7-484d-9923-e43ee09fe454 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:08.513385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45eef2-c2b7-484d-9923-e43ee09fe454 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:08.513733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45eef2-c2b7-484d-9923-e43ee09fe454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:08.573312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45eef2-c2b7-484d-9923-e43ee09fe454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:08.574880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45eef2-c2b7-484d-9923-e43ee09fe454 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:08.584378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1118/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:09.270788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:09.272265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:09.291575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44788e-9a9a-4c4e-a0bb-1529dd0c4840 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:09.295442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b0442-af47-4b3f-b4ff-8d3840724547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:09.296044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b0442-af47-4b3f-b4ff-8d3840724547 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:09.296330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b0442-af47-4b3f-b4ff-8d3840724547 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:09.296581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b0442-af47-4b3f-b4ff-8d3840724547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:09.323902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b0442-af47-4b3f-b4ff-8d3840724547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:09.327236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b0442-af47-4b3f-b4ff-8d3840724547 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:09.356037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1124/2242] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:09.707921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45eef2-c2b7-484d-9923-e43ee09fe454 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:09.771704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45eef2-c2b7-484d-9923-e43ee09fe454 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:09.778491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fea06b-dfe6-4623-b297-3560f14131a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:09.779144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fea06b-dfe6-4623-b297-3560f14131a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:09.779429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fea06b-dfe6-4623-b297-3560f14131a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:09.779700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fea06b-dfe6-4623-b297-3560f14131a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:09.782143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6fea06b-dfe6-4623-b297-3560f14131a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:09.805176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:09.859647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fea06b-dfe6-4623-b297-3560f14131a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:09.861450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fea06b-dfe6-4623-b297-3560f14131a9 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:09.874589 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1119/2243] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:13.744143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b0442-af47-4b3f-b4ff-8d3840724547 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:13.752472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:13.859255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b0442-af47-4b3f-b4ff-8d3840724547 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:13.867038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:13.868281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:13.868573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:13.868917 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] PATH_INFO: `/v3/users/df2a80432f58487c87bba8dff5fe75fb` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:13.871557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] RBAC: Authorizing `identity:delete_user(user_id=df2a80432f58487c87bba8dff5fe75fb)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:13.890828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:13.958383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df2a80432f58487c87bba8dff5fe75fb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:13.967588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df2a80432f58487c87bba8dff5fe75fb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:13.968358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df2a80432f58487c87bba8dff5fe75fb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:13.969024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df2a80432f58487c87bba8dff5fe75fb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:13.979946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df2a80432f58487c87bba8dff5fe75fb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.002981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1125/2244] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:02:13 2022] DELETE /identity/v3/users/df2a80432f58487c87bba8dff5fe75fb => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:14.004797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513c525-0bf1-4ec7-86d0-61abb831a362 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:14.005268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513c525-0bf1-4ec7-86d0-61abb831a362 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:14.005638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513c525-0bf1-4ec7-86d0-61abb831a362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:14.025498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0513c525-0bf1-4ec7-86d0-61abb831a362 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:14.026793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0513c525-0bf1-4ec7-86d0-61abb831a362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:14.089445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1120/2245] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:14.099363 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:14.105093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:14.179114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e6e49-4aeb-4070-a928-7a69433e55db admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:14.184602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e9915-391e-40b1-afa7-602d80207096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:14.185322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e9915-391e-40b1-afa7-602d80207096 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:14.185711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e9915-391e-40b1-afa7-602d80207096 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:14.186182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e9915-391e-40b1-afa7-602d80207096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:14.201200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e9915-391e-40b1-afa7-602d80207096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:14.202877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e9915-391e-40b1-afa7-602d80207096 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:14.213953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1126/2246] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:14.516213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513c525-0bf1-4ec7-86d0-61abb831a362 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:14.517753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0513c525-0bf1-4ec7-86d0-61abb831a362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:14.559025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513c525-0bf1-4ec7-86d0-61abb831a362 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:14.565843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:14.566517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:14.566805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:14.567123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] PATH_INFO: `/v3/projects/4f7d52a627fc46d58caf9161add059cb` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:14.569029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f7d52a627fc46d58caf9161add059cb)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:14.571802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:14.652728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f7d52a627fc46d58caf9161add059cb', 'request_id': 'req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.663221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f7d52a627fc46d58caf9161add059cb', 'request_id': 'req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.671666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f7d52a627fc46d58caf9161add059cb', 'request_id': 'req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.684007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] The token cache is being invalidate because project 4f7d52a627fc46d58caf9161add059cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:02:14.684422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.685102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.686626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1121/2247] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:02:14 2022] DELETE /identity/v3/projects/4f7d52a627fc46d58caf9161add059cb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:02:14.690726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e9915-391e-40b1-afa7-602d80207096 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:14.705135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e9915-391e-40b1-afa7-602d80207096 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:14.730926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:14.769452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:14.770084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:14.770445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:14.770786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] PATH_INFO: `/v3/users/a7a37fd440ba4ec2b91edb2acc3898af` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:14.772086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a37fd440ba4ec2b91edb2acc3898af)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:14.774100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:14.823675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a37fd440ba4ec2b91edb2acc3898af'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.831502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a37fd440ba4ec2b91edb2acc3898af'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.832583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a37fd440ba4ec2b91edb2acc3898af'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.833599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a37fd440ba4ec2b91edb2acc3898af'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.843960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a37fd440ba4ec2b91edb2acc3898af'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:14.879829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1127/2248] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:02:14 2022] DELETE /identity/v3/users/a7a37fd440ba4ec2b91edb2acc3898af => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:15.307288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:15.309760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:15.337033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063de73d-b15c-4ad3-b8e8-7c30c0dc828c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:15.389431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:15.432805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:15.433564 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:15.433934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:15.434369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] PATH_INFO: `/v3/projects/f3d08ff030ad4fbf988bf4cdfafc4473` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:15.436153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d08ff030ad4fbf988bf4cdfafc4473)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:15.438157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:15.491514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d08ff030ad4fbf988bf4cdfafc4473', 'request_id': 'req-bfe3e41a-5323-474d-a15a-b532deaa1c9d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.501339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d08ff030ad4fbf988bf4cdfafc4473', 'request_id': 'req-bfe3e41a-5323-474d-a15a-b532deaa1c9d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.509460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d08ff030ad4fbf988bf4cdfafc4473', 'request_id': 'req-bfe3e41a-5323-474d-a15a-b532deaa1c9d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.521429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] The token cache is being invalidate because project f3d08ff030ad4fbf988bf4cdfafc4473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:02:15.521714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.522843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfe3e41a-5323-474d-a15a-b532deaa1c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.525043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1122/2249] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:02:15 2022] DELETE /identity/v3/projects/f3d08ff030ad4fbf988bf4cdfafc4473 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:15.528792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:15.531107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:15.546203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b9c28-4bae-4d94-baf0-02a0d59b2e27 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:15.616034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:15.616703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:15.616982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:15.617254 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] PATH_INFO: `/v3/users/070aab1569d643a8a7983d601c3eab06` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:15.618393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=070aab1569d643a8a7983d601c3eab06)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:15.620337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:15.693037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '070aab1569d643a8a7983d601c3eab06'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.695159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:15.706265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '070aab1569d643a8a7983d601c3eab06'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.708555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '070aab1569d643a8a7983d601c3eab06'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.709162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '070aab1569d643a8a7983d601c3eab06'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.719857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '070aab1569d643a8a7983d601c3eab06'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:15.740796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1128/2250] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:02:15 2022] DELETE /identity/v3/users/070aab1569d643a8a7983d601c3eab06 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:15.956909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e13e87-5cdb-4db8-a23c-afbf5abb3d6e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:15.957299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e13e87-5cdb-4db8-a23c-afbf5abb3d6e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:15.957592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e13e87-5cdb-4db8-a23c-afbf5abb3d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:15.994354 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-87e13e87-5cdb-4db8-a23c-afbf5abb3d6e None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:15.996054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-87e13e87-5cdb-4db8-a23c-afbf5abb3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:16.076766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1123/2251] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:02:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:16.087514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:16.089561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:16.108231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e13e87-5cdb-4db8-a23c-afbf5abb3d6e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:16.109785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-87e13e87-5cdb-4db8-a23c-afbf5abb3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:16.201384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f064373-c32b-4a35-923d-95ce20ad3bcf admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:16.208542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:16.209054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e13e87-5cdb-4db8-a23c-afbf5abb3d6e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:16.209624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:16.210221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:16.210838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] PATH_INFO: `/v3/projects/c020fa0bee044e349dff4fbb14b1f9e9` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:16.212440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c020fa0bee044e349dff4fbb14b1f9e9)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:16.214269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:16.214775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:16.215080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:16.215352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:16.215789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:16.224429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:16.226040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:16.234691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1124/2252] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:02:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 12:02:16.272433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c020fa0bee044e349dff4fbb14b1f9e9', 'request_id': 'req-bdc41809-9119-459b-8188-010e276bba4b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:16.283049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c020fa0bee044e349dff4fbb14b1f9e9', 'request_id': 'req-bdc41809-9119-459b-8188-010e276bba4b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:16.290531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c020fa0bee044e349dff4fbb14b1f9e9', 'request_id': 'req-bdc41809-9119-459b-8188-010e276bba4b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:16.303213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] The token cache is being invalidate because project c020fa0bee044e349dff4fbb14b1f9e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:02:16.307350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:16.308168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-bdc41809-9119-459b-8188-010e276bba4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:02:16.310615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1129/2253] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:02:16 2022] DELETE /identity/v3/projects/c020fa0bee044e349dff4fbb14b1f9e9 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:02:26.231442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:26.299154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=63,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:36.825518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:36.826930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:36.882386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b922a0-ed71-4883-a16a-e21abb772c0c service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:36.930571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:36.931145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:36.931431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:36.931706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:36.933452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:36.995503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:36.996941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:37.006331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1125/2254] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:44.861968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22949eca-71e2-4413-98a4-6595f60af612 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:44.862475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22949eca-71e2-4413-98a4-6595f60af612 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:44.862653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22949eca-71e2-4413-98a4-6595f60af612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:44.881737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-22949eca-71e2-4413-98a4-6595f60af612 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:44.883612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-22949eca-71e2-4413-98a4-6595f60af612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:44.943604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1130/2255] 10.222.0.52 () {58 vars in 967 bytes} [Mon Oct 10 12:02:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:44.956496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:44.957875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:45.026787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16157f-d530-432d-a9fa-7d35f1a3f4dd service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:45.031735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877036a1-51c3-4966-9859-c614f3368531 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:45.032336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877036a1-51c3-4966-9859-c614f3368531 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:45.032625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877036a1-51c3-4966-9859-c614f3368531 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:45.032903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877036a1-51c3-4966-9859-c614f3368531 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:45.038872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:45.041332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877036a1-51c3-4966-9859-c614f3368531 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:45.043310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877036a1-51c3-4966-9859-c614f3368531 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:45.052076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1126/2256] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:02:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:02:50.366998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22949eca-71e2-4413-98a4-6595f60af612 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:50.369279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-22949eca-71e2-4413-98a4-6595f60af612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:50.446534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22949eca-71e2-4413-98a4-6595f60af612 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:50.452171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d44cff-e681-42ec-9f50-a9e8635caa90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:50.452887 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44cff-e681-42ec-9f50-a9e8635caa90 service placement] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:50.453271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44cff-e681-42ec-9f50-a9e8635caa90 service placement] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:50.453425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44cff-e681-42ec-9f50-a9e8635caa90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:50.455366 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-99d44cff-e681-42ec-9f50-a9e8635caa90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:50.467739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d44cff-e681-42ec-9f50-a9e8635caa90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:50.469314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d44cff-e681-42ec-9f50-a9e8635caa90 service placement] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:50.480278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1131/2257] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:55.048434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:03:00.475536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=57 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:03:02.970141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc76b33-dc5d-4978-940b-5c558965944c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:03:02.970613 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc76b33-dc5d-4978-940b-5c558965944c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:03:02.970728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc76b33-dc5d-4978-940b-5c558965944c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:03:02.989628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-adc76b33-dc5d-4978-940b-5c558965944c None None] MFA Rules not processed for user `2f7735444ca8431ea7906070374dbe67`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:03:02.990846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-adc76b33-dc5d-4978-940b-5c558965944c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:03:03.047379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1127/2258] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:03:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:03:13.040181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=25 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:03:44.435560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174eccf-a37e-44b5-9d99-663fcc004fb0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:03:44.436078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174eccf-a37e-44b5-9d99-663fcc004fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:03:44.436209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174eccf-a37e-44b5-9d99-663fcc004fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:03:44.454741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5174eccf-a37e-44b5-9d99-663fcc004fb0 None None] MFA Rules not processed for user `b051e414038b470691ee3e91b952a87d`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:03:44.455998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5174eccf-a37e-44b5-9d99-663fcc004fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:03:44.509007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1132/2259] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:03:47.554283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50306409-49e6-43bf-be38-7baea37ed35b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:03:47.555325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50306409-49e6-43bf-be38-7baea37ed35b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:03:47.555438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50306409-49e6-43bf-be38-7baea37ed35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:03:47.574536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-50306409-49e6-43bf-be38-7baea37ed35b None None] MFA Rules not processed for user `99b3e727e6fa480c90a054a60b68b8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:03:47.575754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-50306409-49e6-43bf-be38-7baea37ed35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:03:47.631186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1128/2260] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:03:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:03:54.502353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=25 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:03:57.624528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=25 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:03.908524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174eccf-a37e-44b5-9d99-663fcc004fb0 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:03.909942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5174eccf-a37e-44b5-9d99-663fcc004fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:03.922386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174eccf-a37e-44b5-9d99-663fcc004fb0 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:03.926205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef4a5f-f298-439f-a579-28fbfef7f8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:03.926814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef4a5f-f298-439f-a579-28fbfef7f8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:03.927119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef4a5f-f298-439f-a579-28fbfef7f8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:03.927389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef4a5f-f298-439f-a579-28fbfef7f8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:03.929146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1cef4a5f-f298-439f-a579-28fbfef7f8d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:04.004564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef4a5f-f298-439f-a579-28fbfef7f8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:04.006020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef4a5f-f298-439f-a579-28fbfef7f8d1 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:04.017591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1133/2261] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:04:04.895024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50306409-49e6-43bf-be38-7baea37ed35b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:04.896419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-50306409-49e6-43bf-be38-7baea37ed35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:04.952719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50306409-49e6-43bf-be38-7baea37ed35b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:04.956285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:04.956878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:04.957158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:04.957429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:04.959124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:04.971963 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:04.973358 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:04.984853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1129/2262] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:11.393698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6529a1-017f-4323-bd6c-315fb35d2335 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:11.394187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6529a1-017f-4323-bd6c-315fb35d2335 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:11.394286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6529a1-017f-4323-bd6c-315fb35d2335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:11.415415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-8f6529a1-017f-4323-bd6c-315fb35d2335 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:11.416643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8f6529a1-017f-4323-bd6c-315fb35d2335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:11.423905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1134/2263] 10.222.0.52 () {58 vars in 967 bytes} [Mon Oct 10 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:04:11.476343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:11.488510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bc99f-4b8e-4016-aa5c-d7fd039733b1 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:11.493666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:11.494264 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:11.494546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:11.494819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:11.503177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:11.504772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:11.513128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1130/2264] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:04:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:04:13.766498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1b58-5656-470a-9ccd-097341d1634b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:13.767016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1b58-5656-470a-9ccd-097341d1634b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:13.767117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1b58-5656-470a-9ccd-097341d1634b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:13.770158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:13.771660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:13.785386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75aaab-c40b-4b3a-bf8f-3f1286ac2b7e service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:13.787514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d6fd1b58-5656-470a-9ccd-097341d1634b None None] MFA Rules not processed for user `0c0f1d116c004694b33b19d988ac4959`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:13.788691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d6fd1b58-5656-470a-9ccd-097341d1634b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:13.789270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:13.789988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:13.790370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:13.790738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:13.792668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:13.804242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:13.806298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:13.815787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1131/2265] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:04:13.851038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1135/2266] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:04:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:04:16.325355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:16.336659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66825c-7190-456f-81d4-b13edbe180e0 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:16.341588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:16.342184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:16.342463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:16.342735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:16.344584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:16.356613 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:16.358363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:16.369771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1132/2267] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:20.693252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1b58-5656-470a-9ccd-097341d1634b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:20.696037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d6fd1b58-5656-470a-9ccd-097341d1634b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:20.714016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1b58-5656-470a-9ccd-097341d1634b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:20.723024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:20.723862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:20.724242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:20.724757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:20.726755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:20.738773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:20.741242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:20.751815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1136/2268] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:20.902742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:20.904764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:21.004322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5c80-01bd-46a2-b06c-1c06d8ea8f8a service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:21.010852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:21.012085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:21.012529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:21.013103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] PATH_INFO: `/v3/users/0b2c9c8c10ec455f85cd95a813692de5` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:21.014656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2c9c8c10ec455f85cd95a813692de5)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:21.017187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:21.094467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2c9c8c10ec455f85cd95a813692de5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.102825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2c9c8c10ec455f85cd95a813692de5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.104065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2c9c8c10ec455f85cd95a813692de5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.105648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2c9c8c10ec455f85cd95a813692de5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.123128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2c9c8c10ec455f85cd95a813692de5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.126879 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:21.149269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1133/2269] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:04:20 2022] DELETE /identity/v3/users/0b2c9c8c10ec455f85cd95a813692de5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:04:21.359968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:21.380337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d315196-cdb7-4076-94e7-2966f32ebea5 service nova] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:21.449263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:21.449898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:21.450179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:21.450482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] PATH_INFO: `/v3/projects/f2e5245da9a24e679c6e19089e9aa199` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:21.451690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e5245da9a24e679c6e19089e9aa199)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:21.453585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:21.464564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:21.500258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e5245da9a24e679c6e19089e9aa199', 'request_id': 'req-028d1fd4-54d8-4387-988e-0fbea2514388'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.508576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e5245da9a24e679c6e19089e9aa199', 'request_id': 'req-028d1fd4-54d8-4387-988e-0fbea2514388'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.515050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e5245da9a24e679c6e19089e9aa199', 'request_id': 'req-028d1fd4-54d8-4387-988e-0fbea2514388'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.525671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] The token cache is being invalidate because project f2e5245da9a24e679c6e19089e9aa199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:04:21.525952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.526565 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.528051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1137/2270] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:04:21 2022] DELETE /identity/v3/projects/f2e5245da9a24e679c6e19089e9aa199 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:04:21.531992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:21.533411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:21.546610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebb460-a6c6-4a31-95a4-b8757b4f452f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:21.603529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:21.604016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:21.604297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:21.604569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] PATH_INFO: `/v3/users/8fba495fbddf48ee833bdd4fb1fb48cf` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:21.605685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fba495fbddf48ee833bdd4fb1fb48cf)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:21.607680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:21.655936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fba495fbddf48ee833bdd4fb1fb48cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.664345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fba495fbddf48ee833bdd4fb1fb48cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.665104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fba495fbddf48ee833bdd4fb1fb48cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.666460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fba495fbddf48ee833bdd4fb1fb48cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.676293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6ccf73cd-4386-4a35-b2c5-b05a931402fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fba495fbddf48ee833bdd4fb1fb48cf'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:21.697545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1134/2271] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:04:21 2022] DELETE /identity/v3/users/8fba495fbddf48ee833bdd4fb1fb48cf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:04:21.888224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:21.889817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:21.923490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d1fd4-54d8-4387-988e-0fbea2514388 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:21.994422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:21.995094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:21.995460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:21.995743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] PATH_INFO: `/v3/projects/7bd636aa3db54e8c9eb398a31647292a` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:21.996880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd636aa3db54e8c9eb398a31647292a)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:21.998949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:22.044566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd636aa3db54e8c9eb398a31647292a', 'request_id': 'req-95539f99-8222-48aa-bba1-a45713ff252a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:22.051825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd636aa3db54e8c9eb398a31647292a', 'request_id': 'req-95539f99-8222-48aa-bba1-a45713ff252a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:22.058496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd636aa3db54e8c9eb398a31647292a', 'request_id': 'req-95539f99-8222-48aa-bba1-a45713ff252a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:22.077112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] The token cache is being invalidate because project 7bd636aa3db54e8c9eb398a31647292a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:04:22.077401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:22.078047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:22.080923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1138/2272] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:04:21 2022] DELETE /identity/v3/projects/7bd636aa3db54e8c9eb398a31647292a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:04:24.059975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3593cf-8121-4f06-9553-1dc90c7464be None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:24.060436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3593cf-8121-4f06-9553-1dc90c7464be None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:24.060703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3593cf-8121-4f06-9553-1dc90c7464be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:24.083565 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8a3593cf-8121-4f06-9553-1dc90c7464be None None] MFA Rules not processed for user `2f7735444ca8431ea7906070374dbe67`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:24.084792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8a3593cf-8121-4f06-9553-1dc90c7464be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:24.142128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1135/2273] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:04:24.454253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:24.455712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:24.492305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:24.518689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95539f99-8222-48aa-bba1-a45713ff252a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:24.522357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:24.522998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:24.523283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:24.523562 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:24.525956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:24.536569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:24.537906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:24.547353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1139/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:24.938052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3593cf-8121-4f06-9553-1dc90c7464be None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:24.939757 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8a3593cf-8121-4f06-9553-1dc90c7464be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:24.952870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:25.027454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3593cf-8121-4f06-9553-1dc90c7464be None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:25.032479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:25.033110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:25.033391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:25.033664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:25.035413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:25.046682 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:25.048287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:25.058940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1136/2275] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:26.427407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:26.437076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a34d-a603-458c-8c2c-66d83d2490ad service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:26.440804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:26.441435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:26.441710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:26.441980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:26.443738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:26.454100 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:26.455556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:26.464410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1140/2276] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:31.900547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:31.901958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:31.960787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08687c4-79ce-4f7d-8416-3e0c6ec50abb service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:31.965039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:31.965639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:31.965921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:31.966196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:31.967986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:32.031306 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:32.032630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:32.044243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1137/2277] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:04:36.460681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=18 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:42.038141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=77 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:54.380004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:54.381922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:54.474587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003c4f-dd9f-4a6f-8437-e3f589b19b12 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:54.481791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50ee4f-565b-43e8-87f0-6463d5a4d128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:54.482459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ee4f-565b-43e8-87f0-6463d5a4d128 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:54.482737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ee4f-565b-43e8-87f0-6463d5a4d128 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:54.483034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ee4f-565b-43e8-87f0-6463d5a4d128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:54.484761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5c50ee4f-565b-43e8-87f0-6463d5a4d128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:54.497691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50ee4f-565b-43e8-87f0-6463d5a4d128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:54.499454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50ee4f-565b-43e8-87f0-6463d5a4d128 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:54.511895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1141/2278] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:56.192178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:56.193696 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:56.205875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62239ee1-dce5-43d3-9e9c-c8e74aa5fcb5 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:56.218265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:56.218910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:56.219205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:56.219379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3db5d-ea6f-47ff-81d8-6743571d283b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:56.219857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:56.219966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3db5d-ea6f-47ff-81d8-6743571d283b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:56.219966 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3db5d-ea6f-47ff-81d8-6743571d283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:56.221275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:56.238915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-0aa3db5d-ea6f-47ff-81d8-6743571d283b None None] MFA Rules not processed for user `ce6a1b5ca7584f90bfe83b73aff1335a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:56.240093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0aa3db5d-ea6f-47ff-81d8-6743571d283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:56.286717 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:56.288187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:56.298577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1138/2279] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:04:56.300992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1142/2280] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:04:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:04:56.629365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:56.638622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fe9cb-a6fe-44c6-be79-aca3c6a8016e service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:56.642253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:56.642852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:56.643170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:56.643443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:56.645178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:56.655479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:56.656944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:56.665133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1139/2281] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:59.847242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3db5d-ea6f-47ff-81d8-6743571d283b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:59.848927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0aa3db5d-ea6f-47ff-81d8-6743571d283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:59.891663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3db5d-ea6f-47ff-81d8-6743571d283b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:59.897745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:59.898517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:59.898870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:59.899133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] PATH_INFO: `/v3/users/99b3e727e6fa480c90a054a60b68b8d0` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:59.900861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99b3e727e6fa480c90a054a60b68b8d0)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:59.902786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:59.952530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99b3e727e6fa480c90a054a60b68b8d0'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:59.959852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99b3e727e6fa480c90a054a60b68b8d0'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:59.960523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99b3e727e6fa480c90a054a60b68b8d0'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:59.961147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99b3e727e6fa480c90a054a60b68b8d0'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:59.971341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0774f825-cf22-438a-b5ee-abf6a30bf2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99b3e727e6fa480c90a054a60b68b8d0'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:59.990038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1143/2282] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:04:59 2022] DELETE /identity/v3/users/99b3e727e6fa480c90a054a60b68b8d0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:00.191664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:00.206429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b626264-82c7-4c3a-9064-2a98d1435e13 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:00.278300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:00.279823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.279823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.280003 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] PATH_INFO: `/v3/projects/b72dcfa7755e499f8f55ade0fde3d7c7` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.281138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72dcfa7755e499f8f55ade0fde3d7c7)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:00.283383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:00.343937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72dcfa7755e499f8f55ade0fde3d7c7', 'request_id': 'req-56993e87-bee2-4112-ace2-56cdd19ab6bc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:00.351119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72dcfa7755e499f8f55ade0fde3d7c7', 'request_id': 'req-56993e87-bee2-4112-ace2-56cdd19ab6bc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:00.358357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72dcfa7755e499f8f55ade0fde3d7c7', 'request_id': 'req-56993e87-bee2-4112-ace2-56cdd19ab6bc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:00.369933 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] The token cache is being invalidate because project b72dcfa7755e499f8f55ade0fde3d7c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:00.370223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:00.370910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-56993e87-bee2-4112-ace2-56cdd19ab6bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:00.372434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1140/2283] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:00 2022] DELETE /identity/v3/projects/b72dcfa7755e499f8f55ade0fde3d7c7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:00.377421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48eb31-9a61-49ac-8ef5-877660443d91 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.377805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48eb31-9a61-49ac-8ef5-877660443d91 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.378359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48eb31-9a61-49ac-8ef5-877660443d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.386066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:00.401815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-8d48eb31-9a61-49ac-8ef5-877660443d91 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:00.403051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8d48eb31-9a61-49ac-8ef5-877660443d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:00.464066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1144/2284] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:00.471890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6807b-7733-4efd-9c8d-7494e8e0d04b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.472179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6807b-7733-4efd-9c8d-7494e8e0d04b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.472456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6807b-7733-4efd-9c8d-7494e8e0d04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.493223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cda6807b-7733-4efd-9c8d-7494e8e0d04b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:00.495075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cda6807b-7733-4efd-9c8d-7494e8e0d04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:00.503510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1141/2285] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:00.507848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48eb31-9a61-49ac-8ef5-877660443d91 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:00.517555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48eb31-9a61-49ac-8ef5-877660443d91 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:00.522108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:00.522709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.523017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.523293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.523929 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:00.524229 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:00.525850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:00.537441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1145/2286] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:05:00.542381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6807b-7733-4efd-9c8d-7494e8e0d04b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:00.548118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:00.552658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6807b-7733-4efd-9c8d-7494e8e0d04b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:00.559471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc6d13-4fd5-4ec3-bf85-ecfe2b93f774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:00.560259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6d13-4fd5-4ec3-bf85-ecfe2b93f774 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.560693 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6d13-4fd5-4ec3-bf85-ecfe2b93f774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.561137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6d13-4fd5-4ec3-bf85-ecfe2b93f774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.561885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc6d13-4fd5-4ec3-bf85-ecfe2b93f774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:00.563890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc6d13-4fd5-4ec3-bf85-ecfe2b93f774 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:00.594998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1142/2287] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:00 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:00.599307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:00.609674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c57acb-f222-42e6-b9bf-f9085eaee5ea admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:00.670047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969e34d-c220-4351-88c2-6722d0e0f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:00.671199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e34d-c220-4351-88c2-6722d0e0f765 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.671199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e34d-c220-4351-88c2-6722d0e0f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.671199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e34d-c220-4351-88c2-6722d0e0f765 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.671825 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969e34d-c220-4351-88c2-6722d0e0f765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:00.680270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969e34d-c220-4351-88c2-6722d0e0f765 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:00.710443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1146/2288] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:00 2022] POST /identity/v3/users => generated 302 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:00.711872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc6d13-4fd5-4ec3-bf85-ecfe2b93f774 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:00.722195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc6d13-4fd5-4ec3-bf85-ecfe2b93f774 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:00.727113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860972d2-6275-4aa4-bd9c-46d8965a3861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:00.727720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860972d2-6275-4aa4-bd9c-46d8965a3861 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.730298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860972d2-6275-4aa4-bd9c-46d8965a3861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.730298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860972d2-6275-4aa4-bd9c-46d8965a3861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.730298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860972d2-6275-4aa4-bd9c-46d8965a3861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:00.731366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860972d2-6275-4aa4-bd9c-46d8965a3861 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:00.742058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1143/2289] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:00.746853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969e34d-c220-4351-88c2-6722d0e0f765 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:00.757135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969e34d-c220-4351-88c2-6722d0e0f765 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:00.761871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42932961-9e95-42d5-9151-adebe539481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:00.762470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42932961-9e95-42d5-9151-adebe539481f admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.762748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42932961-9e95-42d5-9151-adebe539481f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.763090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42932961-9e95-42d5-9151-adebe539481f admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.763729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42932961-9e95-42d5-9151-adebe539481f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:00.765314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42932961-9e95-42d5-9151-adebe539481f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:00.776356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1147/2290] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:00.780857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860972d2-6275-4aa4-bd9c-46d8965a3861 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:00.791710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860972d2-6275-4aa4-bd9c-46d8965a3861 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:00.796509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9537b6-aff9-45a6-b604-78887607d4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:00.797170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9537b6-aff9-45a6-b604-78887607d4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.797453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9537b6-aff9-45a6-b604-78887607d4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.797728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9537b6-aff9-45a6-b604-78887607d4f7 admin admin] PATH_INFO: `/v3/projects/db8791e989314e6a84204e4483335991/users/8a66114b4ccf49589e8d0e07b4443009/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.814874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9537b6-aff9-45a6-b604-78887607d4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8791e989314e6a84204e4483335991, user_id=8a66114b4ccf49589e8d0e07b4443009, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:00.817673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9537b6-aff9-45a6-b604-78887607d4f7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:00.832019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1144/2291] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:00 2022] PUT /identity/v3/projects/db8791e989314e6a84204e4483335991/users/8a66114b4ccf49589e8d0e07b4443009/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:00.843865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c810-e198-4328-b2b3-be4466ab1a4e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:00.844240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c810-e198-4328-b2b3-be4466ab1a4e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:00.844512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c810-e198-4328-b2b3-be4466ab1a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:00.872787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-25f4c810-e198-4328-b2b3-be4466ab1a4e None None] MFA Rules not processed for user `8a66114b4ccf49589e8d0e07b4443009`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:00.873986 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-25f4c810-e198-4328-b2b3-be4466ab1a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:00.934336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:00.940646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1148/2292] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:00.945142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9537b6-aff9-45a6-b604-78887607d4f7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.005771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9537b6-aff9-45a6-b604-78887607d4f7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:01.010950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f7733-116c-4a8e-a017-a9fe019e6615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:01.011542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f7733-116c-4a8e-a017-a9fe019e6615 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.011826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f7733-116c-4a8e-a017-a9fe019e6615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.012095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f7733-116c-4a8e-a017-a9fe019e6615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.012725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f7733-116c-4a8e-a017-a9fe019e6615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:01.015000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f7733-116c-4a8e-a017-a9fe019e6615 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:01.040570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1145/2293] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:00 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:01.044674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4c810-e198-4328-b2b3-be4466ab1a4e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.055659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4c810-e198-4328-b2b3-be4466ab1a4e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:01.111743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba444cb2-948b-4770-a7c1-8eda353cd1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:01.112324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444cb2-948b-4770-a7c1-8eda353cd1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.112626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444cb2-948b-4770-a7c1-8eda353cd1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.112893 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444cb2-948b-4770-a7c1-8eda353cd1a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.113498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba444cb2-948b-4770-a7c1-8eda353cd1a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:01.115434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba444cb2-948b-4770-a7c1-8eda353cd1a3 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:01.138573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1149/2294] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:01 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:01.142749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f7733-116c-4a8e-a017-a9fe019e6615 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.153104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f7733-116c-4a8e-a017-a9fe019e6615 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:01.157823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47463b85-e987-4548-ba8d-9e3aa1927f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:01.158437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47463b85-e987-4548-ba8d-9e3aa1927f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.158719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47463b85-e987-4548-ba8d-9e3aa1927f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.159021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47463b85-e987-4548-ba8d-9e3aa1927f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.159664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47463b85-e987-4548-ba8d-9e3aa1927f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:01.161239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47463b85-e987-4548-ba8d-9e3aa1927f8e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:01.172492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1146/2295] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:01.176917 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba444cb2-948b-4770-a7c1-8eda353cd1a3 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.187731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba444cb2-948b-4770-a7c1-8eda353cd1a3 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:01.192385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97e447-58e1-459c-9a1b-dd011ac1462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:01.193040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97e447-58e1-459c-9a1b-dd011ac1462a admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.193322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97e447-58e1-459c-9a1b-dd011ac1462a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.193627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97e447-58e1-459c-9a1b-dd011ac1462a admin admin] PATH_INFO: `/v3/projects/a3bfe6f2a120412b8c505f02e8b59afe/users/8644875ec0a0497ab1d85f79b9cebc16/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.209770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97e447-58e1-459c-9a1b-dd011ac1462a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bfe6f2a120412b8c505f02e8b59afe, user_id=8644875ec0a0497ab1d85f79b9cebc16, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:01.212543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97e447-58e1-459c-9a1b-dd011ac1462a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:01.225926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1150/2296] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:01 2022] PUT /identity/v3/projects/a3bfe6f2a120412b8c505f02e8b59afe/users/8644875ec0a0497ab1d85f79b9cebc16/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:01.230387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47463b85-e987-4548-ba8d-9e3aa1927f8e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.240537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47463b85-e987-4548-ba8d-9e3aa1927f8e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:01.298588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b561bd-880e-41dd-8dfa-3a93ca79c3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:01.299236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b561bd-880e-41dd-8dfa-3a93ca79c3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.299768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b561bd-880e-41dd-8dfa-3a93ca79c3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.300040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b561bd-880e-41dd-8dfa-3a93ca79c3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.300673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b561bd-880e-41dd-8dfa-3a93ca79c3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:01.302259 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b561bd-880e-41dd-8dfa-3a93ca79c3aa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:01.312773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1147/2297] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:01.316947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97e447-58e1-459c-9a1b-dd011ac1462a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.328023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97e447-58e1-459c-9a1b-dd011ac1462a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:01.333055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:01.333695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.333969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.334236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] PATH_INFO: `/v3/projects/a3bfe6f2a120412b8c505f02e8b59afe/users/8644875ec0a0497ab1d85f79b9cebc16/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.336112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bfe6f2a120412b8c505f02e8b59afe, user_id=8644875ec0a0497ab1d85f79b9cebc16, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:01.339348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:01.349678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1151/2298] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:01 2022] PUT /identity/v3/projects/a3bfe6f2a120412b8c505f02e8b59afe/users/8644875ec0a0497ab1d85f79b9cebc16/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:01.360830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626851d-d5d9-40e9-9a9a-7ef19b8e1f07 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.361225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626851d-d5d9-40e9-9a9a-7ef19b8e1f07 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.361567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626851d-d5d9-40e9-9a9a-7ef19b8e1f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.390314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3626851d-d5d9-40e9-9a9a-7ef19b8e1f07 None None] MFA Rules not processed for user `8644875ec0a0497ab1d85f79b9cebc16`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:01.391567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3626851d-d5d9-40e9-9a9a-7ef19b8e1f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:01.396737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:01.468017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1148/2299] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:01.480791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.482478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:01.541667 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377a660-0d4c-4e30-9fbd-a68e738c8ea8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:01.545204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3229bb9-3939-4e43-a706-9365440b3eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:01.545815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3229bb9-3939-4e43-a706-9365440b3eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.546096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3229bb9-3939-4e43-a706-9365440b3eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.546373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3229bb9-3939-4e43-a706-9365440b3eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.555852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3229bb9-3939-4e43-a706-9365440b3eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:01.557329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3229bb9-3939-4e43-a706-9365440b3eb3 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:01.566980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1152/2300] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:01 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:01.725891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626851d-d5d9-40e9-9a9a-7ef19b8e1f07 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.787165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626851d-d5d9-40e9-9a9a-7ef19b8e1f07 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:01.792719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:01.793287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.793568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.793871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] PATH_INFO: `/v3/users/8a66114b4ccf49589e8d0e07b4443009` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.795043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a66114b4ccf49589e8d0e07b4443009)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:01.796961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:01.841811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a66114b4ccf49589e8d0e07b4443009'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:01.848730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a66114b4ccf49589e8d0e07b4443009'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:01.849390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a66114b4ccf49589e8d0e07b4443009'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:01.850002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a66114b4ccf49589e8d0e07b4443009'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:01.858254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a66114b4ccf49589e8d0e07b4443009'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:01.876591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1149/2301] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:01 2022] DELETE /identity/v3/users/8a66114b4ccf49589e8d0e07b4443009 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:01.881027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1364b2-c91b-422d-8781-5616305f300a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:01.881457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1364b2-c91b-422d-8781-5616305f300a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:01.881608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1364b2-c91b-422d-8781-5616305f300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:01.901620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-be1364b2-c91b-422d-8781-5616305f300a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:01.903275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-be1364b2-c91b-422d-8781-5616305f300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:01.956616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:01.970543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1153/2302] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:01.980840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:01.982252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:02.009820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:02.043975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95d7bf-e35b-4cc1-883f-c311b5124f6f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:02.047496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4fae2-ecbf-49ff-80cf-604f301a5e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:02.048092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4fae2-ecbf-49ff-80cf-604f301a5e31 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:02.048367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4fae2-ecbf-49ff-80cf-604f301a5e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:02.048634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4fae2-ecbf-49ff-80cf-604f301a5e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:02.057655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4fae2-ecbf-49ff-80cf-604f301a5e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:02.059091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4fae2-ecbf-49ff-80cf-604f301a5e31 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:02.068578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1150/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:02.329391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1364b2-c91b-422d-8781-5616305f300a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:02.330831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-be1364b2-c91b-422d-8781-5616305f300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:02.342691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1364b2-c91b-422d-8781-5616305f300a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:02.347359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:02.347991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:02.348268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:02.348538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] PATH_INFO: `/v3/projects/db8791e989314e6a84204e4483335991` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:02.349652 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db8791e989314e6a84204e4483335991)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:02.351586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:02.397021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db8791e989314e6a84204e4483335991', 'request_id': 'req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.404160 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db8791e989314e6a84204e4483335991', 'request_id': 'req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.410609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db8791e989314e6a84204e4483335991', 'request_id': 'req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.421691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] The token cache is being invalidate because project db8791e989314e6a84204e4483335991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:02.421977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.422605 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.424178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1154/2304] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:02 2022] DELETE /identity/v3/projects/db8791e989314e6a84204e4483335991 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:02.428572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4fae2-ecbf-49ff-80cf-604f301a5e31 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:02.438630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4fae2-ecbf-49ff-80cf-604f301a5e31 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:02.494138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:02.494759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:02.495070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:02.495346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] PATH_INFO: `/v3/users/8644875ec0a0497ab1d85f79b9cebc16` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:02.496453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8644875ec0a0497ab1d85f79b9cebc16)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:02.498359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:02.543304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8644875ec0a0497ab1d85f79b9cebc16'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.550383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8644875ec0a0497ab1d85f79b9cebc16'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.551065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8644875ec0a0497ab1d85f79b9cebc16'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.551668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8644875ec0a0497ab1d85f79b9cebc16'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.559248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7fc66934-569e-4317-a8c5-a3d90ae546c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8644875ec0a0497ab1d85f79b9cebc16'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.576579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1151/2305] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:02 2022] DELETE /identity/v3/users/8644875ec0a0497ab1d85f79b9cebc16 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:02.842815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:02.844251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:02.859198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7ed9d-aa3a-4943-a27e-28fc308703e2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:02.916006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:02.916665 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:02.916897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:02.917167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] PATH_INFO: `/v3/projects/a3bfe6f2a120412b8c505f02e8b59afe` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:02.918261 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3bfe6f2a120412b8c505f02e8b59afe)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:02.920260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:02.964574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3bfe6f2a120412b8c505f02e8b59afe', 'request_id': 'req-a9c5fbad-3094-414a-8eb4-637c8ccceba2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.971691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3bfe6f2a120412b8c505f02e8b59afe', 'request_id': 'req-a9c5fbad-3094-414a-8eb4-637c8ccceba2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.977790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3bfe6f2a120412b8c505f02e8b59afe', 'request_id': 'req-a9c5fbad-3094-414a-8eb4-637c8ccceba2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.987856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] The token cache is being invalidate because project a3bfe6f2a120412b8c505f02e8b59afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:02.988135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.988740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a9c5fbad-3094-414a-8eb4-637c8ccceba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:02.990183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1155/2306] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:02 2022] DELETE /identity/v3/projects/a3bfe6f2a120412b8c505f02e8b59afe => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:02.995730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b563a-2327-4db8-aba2-419154f3e84f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:02.996021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b563a-2327-4db8-aba2-419154f3e84f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:02.996297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b563a-2327-4db8-aba2-419154f3e84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.017219 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5b6b563a-2327-4db8-aba2-419154f3e84f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:03.018412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b6b563a-2327-4db8-aba2-419154f3e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:03.058727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:03.085493 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1152/2307] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:03.093195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0ae0-b676-4158-9e07-ca4e645f0fb5 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.093481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0ae0-b676-4158-9e07-ca4e645f0fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.093753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0ae0-b676-4158-9e07-ca4e645f0fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.108138 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:03.113859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-daae0ae0-b676-4158-9e07-ca4e645f0fb5 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:03.115894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-daae0ae0-b676-4158-9e07-ca4e645f0fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:03.125358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1156/2308] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:03.129730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b563a-2327-4db8-aba2-419154f3e84f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.140067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b563a-2327-4db8-aba2-419154f3e84f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.144636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.145237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.145517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.145787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.146429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:03.146847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.148455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.160815 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1153/2309] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:05:03.165210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae0ae0-b676-4158-9e07-ca4e645f0fb5 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.174634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae0ae0-b676-4158-9e07-ca4e645f0fb5 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.179425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0d94-327e-46b6-8b25-dd33aa56b2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.180028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0d94-327e-46b6-8b25-dd33aa56b2fa admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.180330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0d94-327e-46b6-8b25-dd33aa56b2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.180597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0d94-327e-46b6-8b25-dd33aa56b2fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.181957 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0d94-327e-46b6-8b25-dd33aa56b2fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.183690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0d94-327e-46b6-8b25-dd33aa56b2fa admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.215558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1157/2310] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:03 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:03.216898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.226825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325363d6-b9bf-42c4-aff4-3d176b60eb91 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.281691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fd5e2-0ac9-45e2-a74d-5ab5a9c4bbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.282308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fd5e2-0ac9-45e2-a74d-5ab5a9c4bbd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.282587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fd5e2-0ac9-45e2-a74d-5ab5a9c4bbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.282859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fd5e2-0ac9-45e2-a74d-5ab5a9c4bbd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.283499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fd5e2-0ac9-45e2-a74d-5ab5a9c4bbd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.285382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fd5e2-0ac9-45e2-a74d-5ab5a9c4bbd0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.307580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1154/2311] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:03 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:03.311691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0d94-327e-46b6-8b25-dd33aa56b2fa admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.322033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0d94-327e-46b6-8b25-dd33aa56b2fa admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.326580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f5a37-82ac-451b-846c-3df19c3d24b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.327217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f5a37-82ac-451b-846c-3df19c3d24b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.327499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f5a37-82ac-451b-846c-3df19c3d24b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.327770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f5a37-82ac-451b-846c-3df19c3d24b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.328405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f5a37-82ac-451b-846c-3df19c3d24b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.329982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f5a37-82ac-451b-846c-3df19c3d24b5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.340226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1158/2312] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:03.343996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fd5e2-0ac9-45e2-a74d-5ab5a9c4bbd0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.354089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fd5e2-0ac9-45e2-a74d-5ab5a9c4bbd0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.358640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759b79c-2e1b-4ed4-ad53-1c4b1a18d44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.359254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759b79c-2e1b-4ed4-ad53-1c4b1a18d44c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.359530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759b79c-2e1b-4ed4-ad53-1c4b1a18d44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.359798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759b79c-2e1b-4ed4-ad53-1c4b1a18d44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.360468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759b79c-2e1b-4ed4-ad53-1c4b1a18d44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.362049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759b79c-2e1b-4ed4-ad53-1c4b1a18d44c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.372443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1155/2313] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:03.376467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f5a37-82ac-451b-846c-3df19c3d24b5 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.386698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f5a37-82ac-451b-846c-3df19c3d24b5 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.391216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921be4-c7d5-44ed-b551-c5090b4d55c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.391850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921be4-c7d5-44ed-b551-c5090b4d55c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.392133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921be4-c7d5-44ed-b551-c5090b4d55c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.392412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921be4-c7d5-44ed-b551-c5090b4d55c6 admin admin] PATH_INFO: `/v3/projects/38cda92f169246799d3ff833e5a62ed5/users/56a091f7bd81486b890cb3d1e060a3f1/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.407256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87921be4-c7d5-44ed-b551-c5090b4d55c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cda92f169246799d3ff833e5a62ed5, user_id=56a091f7bd81486b890cb3d1e060a3f1, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.409894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87921be4-c7d5-44ed-b551-c5090b4d55c6 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.424252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1159/2314] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:03 2022] PUT /identity/v3/projects/38cda92f169246799d3ff833e5a62ed5/users/56a091f7bd81486b890cb3d1e060a3f1/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:03.435735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd3afe-4032-49c2-8d11-db47e8b829f6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.436231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd3afe-4032-49c2-8d11-db47e8b829f6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.436340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd3afe-4032-49c2-8d11-db47e8b829f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.464278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f8cd3afe-4032-49c2-8d11-db47e8b829f6 None None] MFA Rules not processed for user `56a091f7bd81486b890cb3d1e060a3f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:03.465465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f8cd3afe-4032-49c2-8d11-db47e8b829f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:03.527849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1156/2315] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:03.532365 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921be4-c7d5-44ed-b551-c5090b4d55c6 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.593130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921be4-c7d5-44ed-b551-c5090b4d55c6 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.597939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce6bde-d0b6-4e28-86b0-0b2bbb539645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.598576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce6bde-d0b6-4e28-86b0-0b2bbb539645 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.598856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce6bde-d0b6-4e28-86b0-0b2bbb539645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.599162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce6bde-d0b6-4e28-86b0-0b2bbb539645 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.599796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce6bde-d0b6-4e28-86b0-0b2bbb539645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.601751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce6bde-d0b6-4e28-86b0-0b2bbb539645 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.626843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1160/2316] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:03 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:03.631525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd3afe-4032-49c2-8d11-db47e8b829f6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.642065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd3afe-4032-49c2-8d11-db47e8b829f6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.681149 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:03.702827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3b14e-a894-424f-a596-e88f1b9f7950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.703469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b14e-a894-424f-a596-e88f1b9f7950 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.703747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b14e-a894-424f-a596-e88f1b9f7950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.704016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b14e-a894-424f-a596-e88f1b9f7950 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.704646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3b14e-a894-424f-a596-e88f1b9f7950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.706557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3b14e-a894-424f-a596-e88f1b9f7950 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.730847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1157/2317] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:03 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:03.735028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce6bde-d0b6-4e28-86b0-0b2bbb539645 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.745702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce6bde-d0b6-4e28-86b0-0b2bbb539645 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.750300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ae45f-90a4-4872-9cd4-b4655bd48383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.750904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ae45f-90a4-4872-9cd4-b4655bd48383 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.751194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ae45f-90a4-4872-9cd4-b4655bd48383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.751472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ae45f-90a4-4872-9cd4-b4655bd48383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.752106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ae45f-90a4-4872-9cd4-b4655bd48383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.753688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ae45f-90a4-4872-9cd4-b4655bd48383 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.764438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1161/2318] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:03.768374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3b14e-a894-424f-a596-e88f1b9f7950 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.778697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3b14e-a894-424f-a596-e88f1b9f7950 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.783304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245564a0-272e-4a50-abd3-d2ce988df89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.783941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245564a0-272e-4a50-abd3-d2ce988df89b admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.784219 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245564a0-272e-4a50-abd3-d2ce988df89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.784490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245564a0-272e-4a50-abd3-d2ce988df89b admin admin] PATH_INFO: `/v3/projects/00e12e31745f4ac59dd8e40afad6fd6b/users/eb74ca7bda38407d837d9bc0e4cf6125/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.799848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245564a0-272e-4a50-abd3-d2ce988df89b admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e12e31745f4ac59dd8e40afad6fd6b, user_id=eb74ca7bda38407d837d9bc0e4cf6125, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.802582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245564a0-272e-4a50-abd3-d2ce988df89b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.815631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1158/2319] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:03 2022] PUT /identity/v3/projects/00e12e31745f4ac59dd8e40afad6fd6b/users/eb74ca7bda38407d837d9bc0e4cf6125/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:03.819663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ae45f-90a4-4872-9cd4-b4655bd48383 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.830340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ae45f-90a4-4872-9cd4-b4655bd48383 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.885552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231eb10-6f85-42ca-8525-4b868b0cbe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.886070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231eb10-6f85-42ca-8525-4b868b0cbe01 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.886354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231eb10-6f85-42ca-8525-4b868b0cbe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.886627 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231eb10-6f85-42ca-8525-4b868b0cbe01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.887303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231eb10-6f85-42ca-8525-4b868b0cbe01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.888931 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231eb10-6f85-42ca-8525-4b868b0cbe01 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.898482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1162/2320] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:03.902627 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245564a0-272e-4a50-abd3-d2ce988df89b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:03.912810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245564a0-272e-4a50-abd3-d2ce988df89b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:03.917479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:03.918120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.918404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.918681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] PATH_INFO: `/v3/projects/00e12e31745f4ac59dd8e40afad6fd6b/users/eb74ca7bda38407d837d9bc0e4cf6125/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.920731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e12e31745f4ac59dd8e40afad6fd6b, user_id=eb74ca7bda38407d837d9bc0e4cf6125, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:03.923381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:03.934429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1159/2321] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:03 2022] PUT /identity/v3/projects/00e12e31745f4ac59dd8e40afad6fd6b/users/eb74ca7bda38407d837d9bc0e4cf6125/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:03.944582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f975-1463-4180-ad36-7874295954e0 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:03.945029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f975-1463-4180-ad36-7874295954e0 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:03.945189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f975-1463-4180-ad36-7874295954e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:03.971200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:03.974112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6915f975-1463-4180-ad36-7874295954e0 None None] MFA Rules not processed for user `eb74ca7bda38407d837d9bc0e4cf6125`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:03.975874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6915f975-1463-4180-ad36-7874295954e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:04.042484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1163/2322] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:04.054128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:04.055847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:04.112212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de1ad5-04f1-4eab-aa61-8b387f7f2491 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:04.115829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99615964-5943-4723-a6ec-dff58919e3a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:04.116415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99615964-5943-4723-a6ec-dff58919e3a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:04.116688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99615964-5943-4723-a6ec-dff58919e3a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:04.116999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99615964-5943-4723-a6ec-dff58919e3a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:04.126773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99615964-5943-4723-a6ec-dff58919e3a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:04.128227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99615964-5943-4723-a6ec-dff58919e3a9 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:04.137544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1160/2323] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:04.166022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915f975-1463-4180-ad36-7874295954e0 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:04.226479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915f975-1463-4180-ad36-7874295954e0 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:04.231620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:04.232244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:04.232589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:04.232781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] PATH_INFO: `/v3/users/56a091f7bd81486b890cb3d1e060a3f1` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:04.233919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a091f7bd81486b890cb3d1e060a3f1)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:04.235853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:04.291259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a091f7bd81486b890cb3d1e060a3f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.298041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a091f7bd81486b890cb3d1e060a3f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.298708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a091f7bd81486b890cb3d1e060a3f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.299598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a091f7bd81486b890cb3d1e060a3f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.308761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a091f7bd81486b890cb3d1e060a3f1'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.326691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1164/2324] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:04 2022] DELETE /identity/v3/users/56a091f7bd81486b890cb3d1e060a3f1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:04.331332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b467-ee0d-4334-b33d-0dc5e91deb69 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:04.331674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b467-ee0d-4334-b33d-0dc5e91deb69 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:04.331949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b467-ee0d-4334-b33d-0dc5e91deb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:04.350393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f672b467-ee0d-4334-b33d-0dc5e91deb69 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:04.351635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f672b467-ee0d-4334-b33d-0dc5e91deb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:04.414180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1161/2325] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:04.424713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:04.426119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:04.446168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:04.483522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af2e4e-ec1b-44fc-afef-1cd086f1731c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:04.487114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:04.487709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:04.487989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:04.488258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:04.497571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:04.499005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:04.508227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1165/2326] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:04.748402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672b467-ee0d-4334-b33d-0dc5e91deb69 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:04.749820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f672b467-ee0d-4334-b33d-0dc5e91deb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:04.770062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:04.771474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:04.795154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:04.801687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278111de-c524-4374-ae9d-063eddc163a0 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:04.807347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:04.808009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:04.808290 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:04.808704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] PATH_INFO: `/v3/projects/38cda92f169246799d3ff833e5a62ed5` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:04.809858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38cda92f169246799d3ff833e5a62ed5)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:04.811800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:04.832277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672b467-ee0d-4334-b33d-0dc5e91deb69 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:04.838609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:04.839380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:04.839798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:04.840152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:04.842013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:04.860026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38cda92f169246799d3ff833e5a62ed5', 'request_id': 'req-7631ea3e-7298-499f-a649-a31b28ea96f3'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.868831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38cda92f169246799d3ff833e5a62ed5', 'request_id': 'req-7631ea3e-7298-499f-a649-a31b28ea96f3'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.875859 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38cda92f169246799d3ff833e5a62ed5', 'request_id': 'req-7631ea3e-7298-499f-a649-a31b28ea96f3'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.886717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] The token cache is being invalidate because project 38cda92f169246799d3ff833e5a62ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:04.887038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.887693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:04.889278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1166/2327] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:04 2022] DELETE /identity/v3/projects/38cda92f169246799d3ff833e5a62ed5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:04.893208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:04.894505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:04.903436 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:04.905038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:04.914402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1162/2328] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:04.958744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ea3e-7298-499f-a649-a31b28ea96f3 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:04.964270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:04.964935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:04.965262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:04.965507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] PATH_INFO: `/v3/users/eb74ca7bda38407d837d9bc0e4cf6125` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:04.966658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb74ca7bda38407d837d9bc0e4cf6125)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:04.968646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:05.019415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:05.021453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb74ca7bda38407d837d9bc0e4cf6125'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.031354 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb74ca7bda38407d837d9bc0e4cf6125'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.032066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb74ca7bda38407d837d9bc0e4cf6125'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.032687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb74ca7bda38407d837d9bc0e4cf6125'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.039678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ffa5d86e-6ba6-410d-be85-a68fd0c4352e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb74ca7bda38407d837d9bc0e4cf6125'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.058550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1167/2329] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:04 2022] DELETE /identity/v3/users/eb74ca7bda38407d837d9bc0e4cf6125 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:05.355520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:05.366014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a79435-7cab-4cb1-8671-fcb7ebc62e21 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:05.426558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:05.427155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.427440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.427716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] PATH_INFO: `/v3/projects/00e12e31745f4ac59dd8e40afad6fd6b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.428846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e12e31745f4ac59dd8e40afad6fd6b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:05.430736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:05.481413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e12e31745f4ac59dd8e40afad6fd6b', 'request_id': 'req-2bae5ea0-5485-44b9-a48d-61efa590b7fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.489572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e12e31745f4ac59dd8e40afad6fd6b', 'request_id': 'req-2bae5ea0-5485-44b9-a48d-61efa590b7fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.496473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e12e31745f4ac59dd8e40afad6fd6b', 'request_id': 'req-2bae5ea0-5485-44b9-a48d-61efa590b7fd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.507399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] The token cache is being invalidate because project 00e12e31745f4ac59dd8e40afad6fd6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:05.507851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.508599 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2bae5ea0-5485-44b9-a48d-61efa590b7fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:05.510255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1163/2330] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:05 2022] DELETE /identity/v3/projects/00e12e31745f4ac59dd8e40afad6fd6b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:05.515260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16f949-951f-472a-8452-952d913cf549 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.515615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16f949-951f-472a-8452-952d913cf549 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.515932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16f949-951f-472a-8452-952d913cf549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.535922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-fb16f949-951f-472a-8452-952d913cf549 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:05.537110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fb16f949-951f-472a-8452-952d913cf549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:05.596898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1168/2331] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:05.605138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ed698-89e3-48f2-9aa5-b1165c1814a4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.605468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ed698-89e3-48f2-9aa5-b1165c1814a4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.605746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ed698-89e3-48f2-9aa5-b1165c1814a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.614289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:05.627866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0f8ed698-89e3-48f2-9aa5-b1165c1814a4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:05.629906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f8ed698-89e3-48f2-9aa5-b1165c1814a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:05.638322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1164/2332] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:05.642536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16f949-951f-472a-8452-952d913cf549 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:05.653448 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16f949-951f-472a-8452-952d913cf549 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:05.658456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:05.659092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.659388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.659656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.660289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:05.660584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:05.662194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:05.673763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1169/2333] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:05:05.678326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ed698-89e3-48f2-9aa5-b1165c1814a4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:05.687530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ed698-89e3-48f2-9aa5-b1165c1814a4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:05.692147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe5f6f-6406-42db-a581-e3177aca2351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:05.692741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe5f6f-6406-42db-a581-e3177aca2351 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.693018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe5f6f-6406-42db-a581-e3177aca2351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.693287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe5f6f-6406-42db-a581-e3177aca2351 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.693936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe5f6f-6406-42db-a581-e3177aca2351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:05.695866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe5f6f-6406-42db-a581-e3177aca2351 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:05.723484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1165/2334] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:05 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:05.727599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:05.737368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6040c-f2f9-48be-a419-bc3060cc0a33 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:05.790688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed323c-3efc-4044-a8b9-0f8562d642c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:05.791307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed323c-3efc-4044-a8b9-0f8562d642c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.791586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed323c-3efc-4044-a8b9-0f8562d642c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.791855 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed323c-3efc-4044-a8b9-0f8562d642c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.792471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed323c-3efc-4044-a8b9-0f8562d642c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:05.794347 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed323c-3efc-4044-a8b9-0f8562d642c1 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:05.816650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1170/2335] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:05 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:05.820926 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe5f6f-6406-42db-a581-e3177aca2351 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:05.831709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe5f6f-6406-42db-a581-e3177aca2351 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:05.836326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a64356-a154-45f9-9579-2ec458f9e5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:05.836920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64356-a154-45f9-9579-2ec458f9e5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.837800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64356-a154-45f9-9579-2ec458f9e5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.838088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64356-a154-45f9-9579-2ec458f9e5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.838729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a64356-a154-45f9-9579-2ec458f9e5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:05.840343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a64356-a154-45f9-9579-2ec458f9e5a4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:05.850022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1166/2336] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:05.853756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed323c-3efc-4044-a8b9-0f8562d642c1 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:05.863239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed323c-3efc-4044-a8b9-0f8562d642c1 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:05.867762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6723e-1875-48aa-8778-e4ce72ab26f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:05.868337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6723e-1875-48aa-8778-e4ce72ab26f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.868639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6723e-1875-48aa-8778-e4ce72ab26f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.868908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6723e-1875-48aa-8778-e4ce72ab26f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.869537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6723e-1875-48aa-8778-e4ce72ab26f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:05.871120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6723e-1875-48aa-8778-e4ce72ab26f6 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:05.881281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1171/2337] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:05.885783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a64356-a154-45f9-9579-2ec458f9e5a4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:05.895663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a64356-a154-45f9-9579-2ec458f9e5a4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:05.900811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93479c0-5474-4762-8ee7-59ef4a9c3a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:05.901442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93479c0-5474-4762-8ee7-59ef4a9c3a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.901720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93479c0-5474-4762-8ee7-59ef4a9c3a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.901991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93479c0-5474-4762-8ee7-59ef4a9c3a3c admin admin] PATH_INFO: `/v3/projects/0355140a5b1a4a34ae0a3022a46f9a11/users/e7e6acdcef90475480ce7ffabb960dc4/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.916314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93479c0-5474-4762-8ee7-59ef4a9c3a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0355140a5b1a4a34ae0a3022a46f9a11, user_id=e7e6acdcef90475480ce7ffabb960dc4, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:05.918994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93479c0-5474-4762-8ee7-59ef4a9c3a3c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:05.932340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1167/2338] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:05 2022] PUT /identity/v3/projects/0355140a5b1a4a34ae0a3022a46f9a11/users/e7e6acdcef90475480ce7ffabb960dc4/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:05.943369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d1f7c-a0bf-4df9-88f7-e227f5689fda None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:05.943658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d1f7c-a0bf-4df9-88f7-e227f5689fda None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:05.943933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d1f7c-a0bf-4df9-88f7-e227f5689fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:05.971730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-548d1f7c-a0bf-4df9-88f7-e227f5689fda None None] MFA Rules not processed for user `e7e6acdcef90475480ce7ffabb960dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:05.972928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-548d1f7c-a0bf-4df9-88f7-e227f5689fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:05.989267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:06.039250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1172/2339] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:06.043925 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93479c0-5474-4762-8ee7-59ef4a9c3a3c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:06.104355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93479c0-5474-4762-8ee7-59ef4a9c3a3c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:06.108939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44decac6-5a15-407f-881c-6ac4a031650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:06.109556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44decac6-5a15-407f-881c-6ac4a031650c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.109836 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44decac6-5a15-407f-881c-6ac4a031650c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.110105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44decac6-5a15-407f-881c-6ac4a031650c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.110731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44decac6-5a15-407f-881c-6ac4a031650c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:06.112644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44decac6-5a15-407f-881c-6ac4a031650c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:06.139801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1168/2340] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:06 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:06.143954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d1f7c-a0bf-4df9-88f7-e227f5689fda None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:06.154401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d1f7c-a0bf-4df9-88f7-e227f5689fda None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:06.210306 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73386a-35b2-467f-97ee-894a5b7a1359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:06.210833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73386a-35b2-467f-97ee-894a5b7a1359 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.211137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73386a-35b2-467f-97ee-894a5b7a1359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.211404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73386a-35b2-467f-97ee-894a5b7a1359 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.212025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73386a-35b2-467f-97ee-894a5b7a1359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:06.213959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73386a-35b2-467f-97ee-894a5b7a1359 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:06.235575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1173/2341] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:06 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:06.239764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44decac6-5a15-407f-881c-6ac4a031650c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:06.251114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44decac6-5a15-407f-881c-6ac4a031650c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:06.255832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bd42a-156e-48ab-9708-42c8e03ffafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:06.256562 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bd42a-156e-48ab-9708-42c8e03ffafb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.256843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bd42a-156e-48ab-9708-42c8e03ffafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.257111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bd42a-156e-48ab-9708-42c8e03ffafb admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.257774 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bd42a-156e-48ab-9708-42c8e03ffafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:06.260130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bd42a-156e-48ab-9708-42c8e03ffafb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:06.279766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1169/2342] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:06.283220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73386a-35b2-467f-97ee-894a5b7a1359 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:06.294298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73386a-35b2-467f-97ee-894a5b7a1359 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:06.299043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837e3e5-3ff9-4e1b-807d-15b294cab67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:06.299965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837e3e5-3ff9-4e1b-807d-15b294cab67e admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.300250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837e3e5-3ff9-4e1b-807d-15b294cab67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.300525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837e3e5-3ff9-4e1b-807d-15b294cab67e admin admin] PATH_INFO: `/v3/projects/54b3cb62d9864ffc81b5a684df805e1f/users/372629b950a74c44bbc7d1e078f0e38e/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.317701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837e3e5-3ff9-4e1b-807d-15b294cab67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b3cb62d9864ffc81b5a684df805e1f, user_id=372629b950a74c44bbc7d1e078f0e38e, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:06.320399 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837e3e5-3ff9-4e1b-807d-15b294cab67e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:06.335566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1174/2343] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:06 2022] PUT /identity/v3/projects/54b3cb62d9864ffc81b5a684df805e1f/users/372629b950a74c44bbc7d1e078f0e38e/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:06.339667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bd42a-156e-48ab-9708-42c8e03ffafb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:06.350075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bd42a-156e-48ab-9708-42c8e03ffafb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:06.425174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa305b-56ca-4e55-a0be-d82baaa80ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:06.426039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa305b-56ca-4e55-a0be-d82baaa80ebb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.426344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa305b-56ca-4e55-a0be-d82baaa80ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.426638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa305b-56ca-4e55-a0be-d82baaa80ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.427310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa305b-56ca-4e55-a0be-d82baaa80ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:06.429203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa305b-56ca-4e55-a0be-d82baaa80ebb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:06.444022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1170/2344] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:05:06.445418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837e3e5-3ff9-4e1b-807d-15b294cab67e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:06.455932 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837e3e5-3ff9-4e1b-807d-15b294cab67e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:06.461252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:06.461894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.462171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.462438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] PATH_INFO: `/v3/projects/54b3cb62d9864ffc81b5a684df805e1f/users/372629b950a74c44bbc7d1e078f0e38e/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.464305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b3cb62d9864ffc81b5a684df805e1f, user_id=372629b950a74c44bbc7d1e078f0e38e, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:06.466958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:06.477416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1175/2345] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:06 2022] PUT /identity/v3/projects/54b3cb62d9864ffc81b5a684df805e1f/users/372629b950a74c44bbc7d1e078f0e38e/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:06.489462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b54a-8900-4d60-8813-28c8599b9604 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.489635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b54a-8900-4d60-8813-28c8599b9604 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.489920 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b54a-8900-4d60-8813-28c8599b9604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.529928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:06.529928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a4d8b54a-8900-4d60-8813-28c8599b9604 None None] MFA Rules not processed for user `372629b950a74c44bbc7d1e078f0e38e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:06.529928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a4d8b54a-8900-4d60-8813-28c8599b9604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:06.608641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1171/2346] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:06 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:06.620582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:06.622349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:06.683477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9476-bb08-418b-be9b-46fcff3cf4a8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:06.687713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbef81-eef3-4d9e-b539-35743fb45f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:06.688322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbef81-eef3-4d9e-b539-35743fb45f6f service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.688624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbef81-eef3-4d9e-b539-35743fb45f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.688888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbef81-eef3-4d9e-b539-35743fb45f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.703223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbef81-eef3-4d9e-b539-35743fb45f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:06.704695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbef81-eef3-4d9e-b539-35743fb45f6f service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:06.715629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1176/2347] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:06 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:06.765389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8b54a-8900-4d60-8813-28c8599b9604 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:06.766776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a4d8b54a-8900-4d60-8813-28c8599b9604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:06.779144 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8b54a-8900-4d60-8813-28c8599b9604 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:06.782803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:06.783430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:06.783714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:06.783988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:06.842030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:06.843519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:06.852527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1172/2348] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:10.206977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbef81-eef3-4d9e-b539-35743fb45f6f service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:10.224175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:10.253924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:10.255789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:10.294763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbef81-eef3-4d9e-b539-35743fb45f6f service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:10.305694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:10.307025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:10.307381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:10.307724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] PATH_INFO: `/v3/users/e7e6acdcef90475480ce7ffabb960dc4` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:10.309415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e6acdcef90475480ce7ffabb960dc4)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:10.312361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:10.355290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4266b-a5d1-4b28-8307-06e6bf6993a7 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:10.360990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649fe47-0088-455d-90c7-316987f9aa68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:10.361597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649fe47-0088-455d-90c7-316987f9aa68 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:10.361872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649fe47-0088-455d-90c7-316987f9aa68 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:10.362165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649fe47-0088-455d-90c7-316987f9aa68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:10.363902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e6acdcef90475480ce7ffabb960dc4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:10.364026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3649fe47-0088-455d-90c7-316987f9aa68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:10.372123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e6acdcef90475480ce7ffabb960dc4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:10.372796 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e6acdcef90475480ce7ffabb960dc4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:10.373425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e6acdcef90475480ce7ffabb960dc4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:10.376741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649fe47-0088-455d-90c7-316987f9aa68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:10.378661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649fe47-0088-455d-90c7-316987f9aa68 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:10.381193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e6acdcef90475480ce7ffabb960dc4'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:10.389695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1173/2349] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:05:10.398755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1177/2350] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:10 2022] DELETE /identity/v3/users/e7e6acdcef90475480ce7ffabb960dc4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:10.403484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe9ef-9488-4873-85b3-52b6fee6ecb7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:10.403769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe9ef-9488-4873-85b3-52b6fee6ecb7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:10.404091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe9ef-9488-4873-85b3-52b6fee6ecb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:10.425829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c31fe9ef-9488-4873-85b3-52b6fee6ecb7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:10.427114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c31fe9ef-9488-4873-85b3-52b6fee6ecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:10.439992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:10.487973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1174/2351] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:10.498655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:10.500135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:10.557120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872402-4f5b-43ba-a8c1-cd5fb519d425 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:10.560779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77eab8-4240-4011-a9fe-a29e4b678860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:10.561373 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77eab8-4240-4011-a9fe-a29e4b678860 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:10.561653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77eab8-4240-4011-a9fe-a29e4b678860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:10.561926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77eab8-4240-4011-a9fe-a29e4b678860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:10.570686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77eab8-4240-4011-a9fe-a29e4b678860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:10.572733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77eab8-4240-4011-a9fe-a29e4b678860 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:10.581976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1178/2352] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:10.901614 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fe9ef-9488-4873-85b3-52b6fee6ecb7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:10.904025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c31fe9ef-9488-4873-85b3-52b6fee6ecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:10.920599 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fe9ef-9488-4873-85b3-52b6fee6ecb7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:10.927560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:10.928196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:10.928480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:10.928834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] PATH_INFO: `/v3/projects/0355140a5b1a4a34ae0a3022a46f9a11` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:10.930290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0355140a5b1a4a34ae0a3022a46f9a11)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:10.932931 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:10.980393 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0355140a5b1a4a34ae0a3022a46f9a11', 'request_id': 'req-2d988396-a6a7-4cdd-9e68-39a629ab0332'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:10.987458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0355140a5b1a4a34ae0a3022a46f9a11', 'request_id': 'req-2d988396-a6a7-4cdd-9e68-39a629ab0332'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:10.993884 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0355140a5b1a4a34ae0a3022a46f9a11', 'request_id': 'req-2d988396-a6a7-4cdd-9e68-39a629ab0332'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.004555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] The token cache is being invalidate because project 0355140a5b1a4a34ae0a3022a46f9a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:11.004840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.005461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.006941 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1175/2353] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:10 2022] DELETE /identity/v3/projects/0355140a5b1a4a34ae0a3022a46f9a11 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:11.011132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77eab8-4240-4011-a9fe-a29e4b678860 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:11.020685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77eab8-4240-4011-a9fe-a29e4b678860 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:11.053264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:11.078931 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:11.079554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:11.079834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:11.080106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] PATH_INFO: `/v3/users/372629b950a74c44bbc7d1e078f0e38e` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:11.081215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] RBAC: Authorizing `identity:delete_user(user_id=372629b950a74c44bbc7d1e078f0e38e)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:11.083170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:11.127211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372629b950a74c44bbc7d1e078f0e38e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.134849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372629b950a74c44bbc7d1e078f0e38e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.135507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372629b950a74c44bbc7d1e078f0e38e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.136112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372629b950a74c44bbc7d1e078f0e38e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.142959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3f22c230-9223-441e-b129-c8f704a0204b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372629b950a74c44bbc7d1e078f0e38e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.160677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1179/2354] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:11 2022] DELETE /identity/v3/users/372629b950a74c44bbc7d1e078f0e38e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:11.487326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:11.489411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:11.504729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d988396-a6a7-4cdd-9e68-39a629ab0332 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:11.567058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:11.567686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:11.567966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:11.568235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] PATH_INFO: `/v3/projects/54b3cb62d9864ffc81b5a684df805e1f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:11.569387 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54b3cb62d9864ffc81b5a684df805e1f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:11.571325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:11.615008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:11.619432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54b3cb62d9864ffc81b5a684df805e1f', 'request_id': 'req-65194b95-ede0-4a71-84f3-4934d0e9ed32'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.627212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54b3cb62d9864ffc81b5a684df805e1f', 'request_id': 'req-65194b95-ede0-4a71-84f3-4934d0e9ed32'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.633519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54b3cb62d9864ffc81b5a684df805e1f', 'request_id': 'req-65194b95-ede0-4a71-84f3-4934d0e9ed32'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.643916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] The token cache is being invalidate because project 54b3cb62d9864ffc81b5a684df805e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:11.644205 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.644828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-65194b95-ede0-4a71-84f3-4934d0e9ed32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:11.646338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1176/2355] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:11 2022] DELETE /identity/v3/projects/54b3cb62d9864ffc81b5a684df805e1f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:11.651234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29680558-b96d-4d51-8730-2bd601ffbb86 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:11.651729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29680558-b96d-4d51-8730-2bd601ffbb86 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:11.651822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29680558-b96d-4d51-8730-2bd601ffbb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:11.671511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-29680558-b96d-4d51-8730-2bd601ffbb86 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:11.672707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-29680558-b96d-4d51-8730-2bd601ffbb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:11.731197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1180/2356] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:11.739745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b69c-c73c-4029-aa9f-a6f65f66b5b7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:11.740035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b69c-c73c-4029-aa9f-a6f65f66b5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:11.740315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b69c-c73c-4029-aa9f-a6f65f66b5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:11.760862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ced8b69c-c73c-4029-aa9f-a6f65f66b5b7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:11.762078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ced8b69c-c73c-4029-aa9f-a6f65f66b5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:11.769904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1177/2357] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:11.773987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29680558-b96d-4d51-8730-2bd601ffbb86 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:11.784158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29680558-b96d-4d51-8730-2bd601ffbb86 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:11.788723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:11.789316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:11.789597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:11.789875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:11.790519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:11.790822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:11.792439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:11.803943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1181/2358] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:05:11.807978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8b69c-c73c-4029-aa9f-a6f65f66b5b7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:11.816981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8b69c-c73c-4029-aa9f-a6f65f66b5b7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:11.822289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f53139-0bc1-4343-86a7-a357967b6aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:11.822877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f53139-0bc1-4343-86a7-a357967b6aaf admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:11.823197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f53139-0bc1-4343-86a7-a357967b6aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:11.823461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f53139-0bc1-4343-86a7-a357967b6aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:11.824088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f53139-0bc1-4343-86a7-a357967b6aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:11.825986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f53139-0bc1-4343-86a7-a357967b6aaf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:11.852906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1178/2359] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:11 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:11.856943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:11.866725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae5b78-642d-431c-8c91-10fd72cf4f0e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:11.906916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:11.928377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3be43-9fd3-45d9-ae19-fca26b5e9143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:11.928377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3be43-9fd3-45d9-ae19-fca26b5e9143 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:11.928377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3be43-9fd3-45d9-ae19-fca26b5e9143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:11.928377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3be43-9fd3-45d9-ae19-fca26b5e9143 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:11.929049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3be43-9fd3-45d9-ae19-fca26b5e9143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:11.931280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3be43-9fd3-45d9-ae19-fca26b5e9143 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:11.955386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1182/2360] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:11 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:11.959455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f53139-0bc1-4343-86a7-a357967b6aaf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:11.972311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f53139-0bc1-4343-86a7-a357967b6aaf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:11.977229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaca4cf-524e-4acc-af4a-68e8a7a9bfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:11.977858 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca4cf-524e-4acc-af4a-68e8a7a9bfb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:11.978138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca4cf-524e-4acc-af4a-68e8a7a9bfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:11.978410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca4cf-524e-4acc-af4a-68e8a7a9bfb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:11.979084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaca4cf-524e-4acc-af4a-68e8a7a9bfb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:11.980801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaca4cf-524e-4acc-af4a-68e8a7a9bfb4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:11.995246 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1179/2361] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:11.996411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3be43-9fd3-45d9-ae19-fca26b5e9143 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.006660 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3be43-9fd3-45d9-ae19-fca26b5e9143 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.011817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b201df-3837-4749-b1d6-515731b58001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.012446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b201df-3837-4749-b1d6-515731b58001 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.012730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b201df-3837-4749-b1d6-515731b58001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.012995 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b201df-3837-4749-b1d6-515731b58001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.013614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b201df-3837-4749-b1d6-515731b58001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.015258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b201df-3837-4749-b1d6-515731b58001 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.026768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1183/2362] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:12.030865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaca4cf-524e-4acc-af4a-68e8a7a9bfb4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.041804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaca4cf-524e-4acc-af4a-68e8a7a9bfb4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.046679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0f070-cfcc-44d2-bf8f-6fb166b32f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.047402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0f070-cfcc-44d2-bf8f-6fb166b32f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.047689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0f070-cfcc-44d2-bf8f-6fb166b32f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.047965 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0f070-cfcc-44d2-bf8f-6fb166b32f73 admin admin] PATH_INFO: `/v3/projects/61ad6160efc84ee78406b19c4d537c5d/users/6a9ad31e1a9648a9acd5032b6571f90f/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.063445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0f070-cfcc-44d2-bf8f-6fb166b32f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ad6160efc84ee78406b19c4d537c5d, user_id=6a9ad31e1a9648a9acd5032b6571f90f, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.066120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0f070-cfcc-44d2-bf8f-6fb166b32f73 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.079057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1180/2363] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:12 2022] PUT /identity/v3/projects/61ad6160efc84ee78406b19c4d537c5d/users/6a9ad31e1a9648a9acd5032b6571f90f/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:12.089834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ac4fa-a032-4e60-b841-e93daca0a2c7 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.090121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ac4fa-a032-4e60-b841-e93daca0a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.090395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ac4fa-a032-4e60-b841-e93daca0a2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.119800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-c15ac4fa-a032-4e60-b841-e93daca0a2c7 None None] MFA Rules not processed for user `6a9ad31e1a9648a9acd5032b6571f90f`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:12.121009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-c15ac4fa-a032-4e60-b841-e93daca0a2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:12.183593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1184/2364] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:12.188045 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0f070-cfcc-44d2-bf8f-6fb166b32f73 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.252653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0f070-cfcc-44d2-bf8f-6fb166b32f73 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.257385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4643ac-ae0f-496e-98ed-3162e44ed098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.257988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4643ac-ae0f-496e-98ed-3162e44ed098 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.258267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4643ac-ae0f-496e-98ed-3162e44ed098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.258537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4643ac-ae0f-496e-98ed-3162e44ed098 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.259234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4643ac-ae0f-496e-98ed-3162e44ed098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.261127 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4643ac-ae0f-496e-98ed-3162e44ed098 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.287829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1181/2365] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:12 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:12.292518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ac4fa-a032-4e60-b841-e93daca0a2c7 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.303361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ac4fa-a032-4e60-b841-e93daca0a2c7 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.361149 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8eff9-1228-404f-8dd6-37fe3ad32a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.361792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8eff9-1228-404f-8dd6-37fe3ad32a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.362098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8eff9-1228-404f-8dd6-37fe3ad32a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.362425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8eff9-1228-404f-8dd6-37fe3ad32a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.363085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8eff9-1228-404f-8dd6-37fe3ad32a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.365020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8eff9-1228-404f-8dd6-37fe3ad32a0e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.387259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1185/2366] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:12 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:12.391220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4643ac-ae0f-496e-98ed-3162e44ed098 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.403022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4643ac-ae0f-496e-98ed-3162e44ed098 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.408536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e7777-f9bb-467b-a4b7-8acfbe25e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.409327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e7777-f9bb-467b-a4b7-8acfbe25e22d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.409615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e7777-f9bb-467b-a4b7-8acfbe25e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.409888 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e7777-f9bb-467b-a4b7-8acfbe25e22d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.410567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e7777-f9bb-467b-a4b7-8acfbe25e22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.412230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e7777-f9bb-467b-a4b7-8acfbe25e22d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.423720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1182/2367] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:12.428169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8eff9-1228-404f-8dd6-37fe3ad32a0e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.438032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8eff9-1228-404f-8dd6-37fe3ad32a0e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.442617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d36c2-b744-4bf3-a0af-8410d9e83400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.443238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d36c2-b744-4bf3-a0af-8410d9e83400 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.443518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d36c2-b744-4bf3-a0af-8410d9e83400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.443783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d36c2-b744-4bf3-a0af-8410d9e83400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.444413 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d36c2-b744-4bf3-a0af-8410d9e83400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.445989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d36c2-b744-4bf3-a0af-8410d9e83400 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.456114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1186/2368] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:12.460079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e7777-f9bb-467b-a4b7-8acfbe25e22d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.470773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e7777-f9bb-467b-a4b7-8acfbe25e22d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.475410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30771a-b898-485b-9c93-aa69b83f97c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.476046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30771a-b898-485b-9c93-aa69b83f97c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.476327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30771a-b898-485b-9c93-aa69b83f97c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.476604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30771a-b898-485b-9c93-aa69b83f97c8 admin admin] PATH_INFO: `/v3/projects/8db9086707f241b1993a962be1d1d5bb/users/cd5ffdd9982345c4a2f40cf9629a4f1c/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.491818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30771a-b898-485b-9c93-aa69b83f97c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db9086707f241b1993a962be1d1d5bb, user_id=cd5ffdd9982345c4a2f40cf9629a4f1c, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.494467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30771a-b898-485b-9c93-aa69b83f97c8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.507862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1183/2369] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:12 2022] PUT /identity/v3/projects/8db9086707f241b1993a962be1d1d5bb/users/cd5ffdd9982345c4a2f40cf9629a4f1c/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:12.518598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c331d33-a4ab-46bb-ac86-0c9da09bc7b3 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.518906 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c331d33-a4ab-46bb-ac86-0c9da09bc7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.519197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c331d33-a4ab-46bb-ac86-0c9da09bc7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.546898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-2c331d33-a4ab-46bb-ac86-0c9da09bc7b3 None None] MFA Rules not processed for user `cd5ffdd9982345c4a2f40cf9629a4f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:12.548193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2c331d33-a4ab-46bb-ac86-0c9da09bc7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:12.565298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:12.614656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1187/2370] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:12.619171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30771a-b898-485b-9c93-aa69b83f97c8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.639158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:12.684765 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30771a-b898-485b-9c93-aa69b83f97c8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.689478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0a416-5e7c-4a9a-9194-98bff7520a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.690088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0a416-5e7c-4a9a-9194-98bff7520a91 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.690369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0a416-5e7c-4a9a-9194-98bff7520a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.690645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0a416-5e7c-4a9a-9194-98bff7520a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.691302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0a416-5e7c-4a9a-9194-98bff7520a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.693200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0a416-5e7c-4a9a-9194-98bff7520a91 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.721748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1184/2371] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:12 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:12.725900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c331d33-a4ab-46bb-ac86-0c9da09bc7b3 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.737516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c331d33-a4ab-46bb-ac86-0c9da09bc7b3 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.794080 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0187-e7cc-455f-bb40-3a48b9629300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.794602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0187-e7cc-455f-bb40-3a48b9629300 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.794880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0187-e7cc-455f-bb40-3a48b9629300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.795178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db0187-e7cc-455f-bb40-3a48b9629300 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.795816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db0187-e7cc-455f-bb40-3a48b9629300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.797733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db0187-e7cc-455f-bb40-3a48b9629300 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.820400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1188/2372] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:12 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:12.824952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0a416-5e7c-4a9a-9194-98bff7520a91 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.835603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0a416-5e7c-4a9a-9194-98bff7520a91 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.840286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9e802-bab1-449c-a5c7-e43ee7632750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.840888 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9e802-bab1-449c-a5c7-e43ee7632750 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.841168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9e802-bab1-449c-a5c7-e43ee7632750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.841447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9e802-bab1-449c-a5c7-e43ee7632750 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.842208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9e802-bab1-449c-a5c7-e43ee7632750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.843835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9e802-bab1-449c-a5c7-e43ee7632750 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.854584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1185/2373] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:12.858857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0187-e7cc-455f-bb40-3a48b9629300 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.869887 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db0187-e7cc-455f-bb40-3a48b9629300 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.874657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef266b6f-21e9-4180-b3f2-8854829dd8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.875372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266b6f-21e9-4180-b3f2-8854829dd8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.875655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266b6f-21e9-4180-b3f2-8854829dd8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.875930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266b6f-21e9-4180-b3f2-8854829dd8e5 admin admin] PATH_INFO: `/v3/projects/713bd904cfeb4312ad61fac48fb58b4e/users/f66244b323534753a4cce7b47b4c3743/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.892930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef266b6f-21e9-4180-b3f2-8854829dd8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713bd904cfeb4312ad61fac48fb58b4e, user_id=f66244b323534753a4cce7b47b4c3743, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.895608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef266b6f-21e9-4180-b3f2-8854829dd8e5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:12.909678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1189/2374] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:12 2022] PUT /identity/v3/projects/713bd904cfeb4312ad61fac48fb58b4e/users/f66244b323534753a4cce7b47b4c3743/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:12.913679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9e802-bab1-449c-a5c7-e43ee7632750 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:12.924512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9e802-bab1-449c-a5c7-e43ee7632750 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:12.984462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786449af-8d64-48fa-a136-05284fbe0fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:12.985077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786449af-8d64-48fa-a136-05284fbe0fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:12.985361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786449af-8d64-48fa-a136-05284fbe0fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:12.985635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786449af-8d64-48fa-a136-05284fbe0fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:12.986276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786449af-8d64-48fa-a136-05284fbe0fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:12.987890 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786449af-8d64-48fa-a136-05284fbe0fc0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:13.001213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1186/2375] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:05:13.002339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef266b6f-21e9-4180-b3f2-8854829dd8e5 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:13.013245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef266b6f-21e9-4180-b3f2-8854829dd8e5 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:13.018225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:13.018875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:13.019179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:13.019454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] PATH_INFO: `/v3/projects/713bd904cfeb4312ad61fac48fb58b4e/users/f66244b323534753a4cce7b47b4c3743/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:13.021343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713bd904cfeb4312ad61fac48fb58b4e, user_id=f66244b323534753a4cce7b47b4c3743, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:13.023985 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:13.035054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1190/2376] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:13 2022] PUT /identity/v3/projects/713bd904cfeb4312ad61fac48fb58b4e/users/f66244b323534753a4cce7b47b4c3743/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:13.046226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0ea95-3802-4bb8-a4b8-097159c37d43 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:13.046448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0ea95-3802-4bb8-a4b8-097159c37d43 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:13.046720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0ea95-3802-4bb8-a4b8-097159c37d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:13.076417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-81e0ea95-3802-4bb8-a4b8-097159c37d43 None None] MFA Rules not processed for user `f66244b323534753a4cce7b47b4c3743`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:13.077647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-81e0ea95-3802-4bb8-a4b8-097159c37d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:13.148490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1187/2377] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:13.160745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:13.162725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:13.246211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d137c-89ab-4e2f-928f-2d7c6b1ca097 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:13.250277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:13.250884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:13.251189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:13.251456 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:13.262188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:13.264158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:13.277510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1191/2378] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:13.337678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0ea95-3802-4bb8-a4b8-097159c37d43 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:13.348918 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0ea95-3802-4bb8-a4b8-097159c37d43 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:13.353405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ba7e3-c4bd-417f-9960-fddd10cb6c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:13.354013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ba7e3-c4bd-417f-9960-fddd10cb6c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:13.354298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ba7e3-c4bd-417f-9960-fddd10cb6c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:13.354572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ba7e3-c4bd-417f-9960-fddd10cb6c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:13.364382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:13.423303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ba7e3-c4bd-417f-9960-fddd10cb6c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:13.425017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ba7e3-c4bd-417f-9960-fddd10cb6c8d service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:13.434631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1188/2379] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:14.729391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:14.731065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:14.745862 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:14.809006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee773-ece1-44db-ade4-439ba9c7a843 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:14.814860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf62548-ad48-43d6-b972-6417aa43e6f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:14.815696 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf62548-ad48-43d6-b972-6417aa43e6f0 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:14.815977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf62548-ad48-43d6-b972-6417aa43e6f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:14.816246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf62548-ad48-43d6-b972-6417aa43e6f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:14.819257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-acf62548-ad48-43d6-b972-6417aa43e6f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:14.835827 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf62548-ad48-43d6-b972-6417aa43e6f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:14.838599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf62548-ad48-43d6-b972-6417aa43e6f0 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:14.849385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1192/2380] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:16.367651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ba7e3-c4bd-417f-9960-fddd10cb6c8d service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:16.432164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ba7e3-c4bd-417f-9960-fddd10cb6c8d service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:16.436824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:16.437441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:16.437719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:16.438015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] PATH_INFO: `/v3/users/6a9ad31e1a9648a9acd5032b6571f90f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:16.439142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9ad31e1a9648a9acd5032b6571f90f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:16.441053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:16.486284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9ad31e1a9648a9acd5032b6571f90f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:16.493042 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9ad31e1a9648a9acd5032b6571f90f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:16.494194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9ad31e1a9648a9acd5032b6571f90f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:16.494795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9ad31e1a9648a9acd5032b6571f90f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:16.501894 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9ad31e1a9648a9acd5032b6571f90f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:16.519397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1189/2381] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:16 2022] DELETE /identity/v3/users/6a9ad31e1a9648a9acd5032b6571f90f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:16.523938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7652b7-c903-4f87-aa85-908f8f7e8525 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:16.524419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7652b7-c903-4f87-aa85-908f8f7e8525 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:16.524517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7652b7-c903-4f87-aa85-908f8f7e8525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:16.542848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bb7652b7-c903-4f87-aa85-908f8f7e8525 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:16.544002 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bb7652b7-c903-4f87-aa85-908f8f7e8525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:16.601189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1193/2382] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:16.610828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:16.612324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:16.649606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:16.677053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cece8-1e3a-415a-acf2-afc20b6ce5ec admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:16.681318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71caddb-02b8-433a-87c8-8e189d5369d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:16.681921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71caddb-02b8-433a-87c8-8e189d5369d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:16.682910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71caddb-02b8-433a-87c8-8e189d5369d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:16.683193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71caddb-02b8-433a-87c8-8e189d5369d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:16.692553 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71caddb-02b8-433a-87c8-8e189d5369d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:16.693962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71caddb-02b8-433a-87c8-8e189d5369d4 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:16.703871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1190/2383] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:16.994998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7652b7-c903-4f87-aa85-908f8f7e8525 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:16.996439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bb7652b7-c903-4f87-aa85-908f8f7e8525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:17.009446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7652b7-c903-4f87-aa85-908f8f7e8525 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:17.014462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:17.015299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:17.015461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:17.015695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] PATH_INFO: `/v3/projects/61ad6160efc84ee78406b19c4d537c5d` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:17.017028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ad6160efc84ee78406b19c4d537c5d)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:17.019409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:17.068500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ad6160efc84ee78406b19c4d537c5d', 'request_id': 'req-4516bc27-d842-40a9-be5d-b39dd595822c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.077249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ad6160efc84ee78406b19c4d537c5d', 'request_id': 'req-4516bc27-d842-40a9-be5d-b39dd595822c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.085315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ad6160efc84ee78406b19c4d537c5d', 'request_id': 'req-4516bc27-d842-40a9-be5d-b39dd595822c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.096813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] The token cache is being invalidate because project 61ad6160efc84ee78406b19c4d537c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:17.097099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.097736 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.099316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1194/2384] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:16 2022] DELETE /identity/v3/projects/61ad6160efc84ee78406b19c4d537c5d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:17.103319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71caddb-02b8-433a-87c8-8e189d5369d4 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:17.112965 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71caddb-02b8-433a-87c8-8e189d5369d4 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:17.169234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:17.169862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:17.170141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:17.170418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] PATH_INFO: `/v3/users/cd5ffdd9982345c4a2f40cf9629a4f1c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:17.171581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5ffdd9982345c4a2f40cf9629a4f1c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:17.173496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:17.221969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5ffdd9982345c4a2f40cf9629a4f1c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.229587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5ffdd9982345c4a2f40cf9629a4f1c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.230266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5ffdd9982345c4a2f40cf9629a4f1c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.230870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5ffdd9982345c4a2f40cf9629a4f1c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.239452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5ffdd9982345c4a2f40cf9629a4f1c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.259449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1191/2385] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:17 2022] DELETE /identity/v3/users/cd5ffdd9982345c4a2f40cf9629a4f1c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:17.542026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:17.543548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:17.558033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516bc27-d842-40a9-be5d-b39dd595822c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:17.577576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:17.620031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:17.620682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:17.620967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:17.621244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] PATH_INFO: `/v3/projects/8db9086707f241b1993a962be1d1d5bb` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:17.622412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db9086707f241b1993a962be1d1d5bb)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:17.624449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:17.669434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db9086707f241b1993a962be1d1d5bb', 'request_id': 'req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.676252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db9086707f241b1993a962be1d1d5bb', 'request_id': 'req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.683045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db9086707f241b1993a962be1d1d5bb', 'request_id': 'req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.693038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] The token cache is being invalidate because project 8db9086707f241b1993a962be1d1d5bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:17.693320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.693940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.695450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1195/2386] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:17 2022] DELETE /identity/v3/projects/8db9086707f241b1993a962be1d1d5bb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:17.699360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:17.700724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:17.713101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffe787-743c-426c-a981-fef6a6ff50c3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:17.761304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:17.771756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:17.772401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:17.772676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:17.772948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] PATH_INFO: `/v3/users/f66244b323534753a4cce7b47b4c3743` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:17.774070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66244b323534753a4cce7b47b4c3743)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:17.776035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:17.822389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66244b323534753a4cce7b47b4c3743'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.829468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66244b323534753a4cce7b47b4c3743'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.830047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66244b323534753a4cce7b47b4c3743'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.830643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66244b323534753a4cce7b47b4c3743'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.837839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d0c88965-6f0c-448b-bb7f-ed77593de5bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66244b323534753a4cce7b47b4c3743'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:17.856110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1192/2387] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:17 2022] DELETE /identity/v3/users/f66244b323534753a4cce7b47b4c3743 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:18.157633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.159072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:18.174829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca12a5a-adae-49ac-a9ff-10cbd63e388b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:18.232112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:18.232741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.233017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.233288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] PATH_INFO: `/v3/projects/713bd904cfeb4312ad61fac48fb58b4e` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.234378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=713bd904cfeb4312ad61fac48fb58b4e)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:18.236361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:18.280207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '713bd904cfeb4312ad61fac48fb58b4e', 'request_id': 'req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:18.283712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:18.288388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '713bd904cfeb4312ad61fac48fb58b4e', 'request_id': 'req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:18.296310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '713bd904cfeb4312ad61fac48fb58b4e', 'request_id': 'req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:18.307013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] The token cache is being invalidate because project 713bd904cfeb4312ad61fac48fb58b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:18.307296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:18.307912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-38f0fdb1-b2d2-4379-8fa7-95c91fd587ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:18.309356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1196/2388] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:18 2022] DELETE /identity/v3/projects/713bd904cfeb4312ad61fac48fb58b4e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:18.315271 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f15c1c-fee2-424a-b598-345ab88b886e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.315687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f15c1c-fee2-424a-b598-345ab88b886e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.315964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f15c1c-fee2-424a-b598-345ab88b886e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.335616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-63f15c1c-fee2-424a-b598-345ab88b886e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:18.336637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-63f15c1c-fee2-424a-b598-345ab88b886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:18.394863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1193/2389] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:18.403590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a27c3-b819-42b7-ac60-04ce26d71f5f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.403879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a27c3-b819-42b7-ac60-04ce26d71f5f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.404153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a27c3-b819-42b7-ac60-04ce26d71f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.423689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-993a27c3-b819-42b7-ac60-04ce26d71f5f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:18.425132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-993a27c3-b819-42b7-ac60-04ce26d71f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:18.433029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1197/2390] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:18.437225 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f15c1c-fee2-424a-b598-345ab88b886e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.447386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f15c1c-fee2-424a-b598-345ab88b886e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:18.452151 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:18.452745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.453032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.453296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.453965 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:18.454267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:18.455962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:18.468011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1194/2391] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:05:18.472528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a27c3-b819-42b7-ac60-04ce26d71f5f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.487083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a27c3-b819-42b7-ac60-04ce26d71f5f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:18.487083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436db045-3252-46eb-a59d-336548365dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:18.487083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436db045-3252-46eb-a59d-336548365dec admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.487385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436db045-3252-46eb-a59d-336548365dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.487529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436db045-3252-46eb-a59d-336548365dec admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.488159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436db045-3252-46eb-a59d-336548365dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:18.490044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436db045-3252-46eb-a59d-336548365dec admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:18.521783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1198/2392] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:18 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:18.523265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.533750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913f4d9-185d-45b0-b589-e519eddc771c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:18.588706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c82faf-cb47-42e3-a7fa-db429177fb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:18.589292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c82faf-cb47-42e3-a7fa-db429177fb86 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.589567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c82faf-cb47-42e3-a7fa-db429177fb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.589832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c82faf-cb47-42e3-a7fa-db429177fb86 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.590434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c82faf-cb47-42e3-a7fa-db429177fb86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:18.592346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c82faf-cb47-42e3-a7fa-db429177fb86 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:18.600131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:18.614574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1195/2393] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:18 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:18.618580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436db045-3252-46eb-a59d-336548365dec admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.629272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436db045-3252-46eb-a59d-336548365dec admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:18.634292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a719a9a-a30d-4ab6-a172-9dd81fe39f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:18.634881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a719a9a-a30d-4ab6-a172-9dd81fe39f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.635190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a719a9a-a30d-4ab6-a172-9dd81fe39f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.635452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a719a9a-a30d-4ab6-a172-9dd81fe39f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.636077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a719a9a-a30d-4ab6-a172-9dd81fe39f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:18.637656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a719a9a-a30d-4ab6-a172-9dd81fe39f22 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:18.647393 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1199/2394] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:18.651773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c82faf-cb47-42e3-a7fa-db429177fb86 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.661468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c82faf-cb47-42e3-a7fa-db429177fb86 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:18.666057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a008ea-eb62-4831-aa53-368a5ff26a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:18.666943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a008ea-eb62-4831-aa53-368a5ff26a1f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.667223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a008ea-eb62-4831-aa53-368a5ff26a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.667489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a008ea-eb62-4831-aa53-368a5ff26a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.668111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a008ea-eb62-4831-aa53-368a5ff26a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:18.669664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a008ea-eb62-4831-aa53-368a5ff26a1f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:18.680200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1196/2395] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:05:18.684125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a719a9a-a30d-4ab6-a172-9dd81fe39f22 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.693760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a719a9a-a30d-4ab6-a172-9dd81fe39f22 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:18.698870 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cdecc-e563-4646-a90c-d5a84175d9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:18.699531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cdecc-e563-4646-a90c-d5a84175d9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.699812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cdecc-e563-4646-a90c-d5a84175d9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.700083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cdecc-e563-4646-a90c-d5a84175d9ac admin admin] PATH_INFO: `/v3/projects/22dd25dc1f5c4041b366be4b892333a5/users/f50977ff800b4c15b78da87a139ef3ee/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.714840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cdecc-e563-4646-a90c-d5a84175d9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dd25dc1f5c4041b366be4b892333a5, user_id=f50977ff800b4c15b78da87a139ef3ee, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:18.717551 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cdecc-e563-4646-a90c-d5a84175d9ac admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:18.730423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1200/2396] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:18 2022] PUT /identity/v3/projects/22dd25dc1f5c4041b366be4b892333a5/users/f50977ff800b4c15b78da87a139ef3ee/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:18.742801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da10e1a-af67-4000-9640-3190dd3133d2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.743851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da10e1a-af67-4000-9640-3190dd3133d2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.743967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da10e1a-af67-4000-9640-3190dd3133d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.771450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8da10e1a-af67-4000-9640-3190dd3133d2 None None] MFA Rules not processed for user `f50977ff800b4c15b78da87a139ef3ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:18.772635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8da10e1a-af67-4000-9640-3190dd3133d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:18.835524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1197/2397] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:18.839955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cdecc-e563-4646-a90c-d5a84175d9ac admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.900400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cdecc-e563-4646-a90c-d5a84175d9ac admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:18.904880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdece1-30bc-45ac-bea2-6d24971d4478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:18.905466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdece1-30bc-45ac-bea2-6d24971d4478 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:18.905742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdece1-30bc-45ac-bea2-6d24971d4478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:18.906011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdece1-30bc-45ac-bea2-6d24971d4478 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:18.906629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdece1-30bc-45ac-bea2-6d24971d4478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:18.908548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdece1-30bc-45ac-bea2-6d24971d4478 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:18.934840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1201/2398] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:18 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:18.938856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da10e1a-af67-4000-9640-3190dd3133d2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:18.950024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da10e1a-af67-4000-9640-3190dd3133d2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:19.005049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ca105-b031-4453-8b0d-f07ac86c02cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:19.005656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ca105-b031-4453-8b0d-f07ac86c02cf admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:19.005937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ca105-b031-4453-8b0d-f07ac86c02cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:19.006211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ca105-b031-4453-8b0d-f07ac86c02cf admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:19.006838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ca105-b031-4453-8b0d-f07ac86c02cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:19.008756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ca105-b031-4453-8b0d-f07ac86c02cf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:19.029521 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1198/2399] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:18 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:19.033554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdece1-30bc-45ac-bea2-6d24971d4478 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:19.044618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdece1-30bc-45ac-bea2-6d24971d4478 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:19.049557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda53283-48d3-48c3-881d-e557ffee4ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:19.050154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda53283-48d3-48c3-881d-e557ffee4ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:19.050432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda53283-48d3-48c3-881d-e557ffee4ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:19.050702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda53283-48d3-48c3-881d-e557ffee4ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:19.051369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda53283-48d3-48c3-881d-e557ffee4ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:19.052944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda53283-48d3-48c3-881d-e557ffee4ad9 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:19.063269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1202/2400] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:19.067275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ca105-b031-4453-8b0d-f07ac86c02cf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:19.077194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ca105-b031-4453-8b0d-f07ac86c02cf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:19.081731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8481-8b06-4645-9c06-9e12c0931375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:19.082359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8481-8b06-4645-9c06-9e12c0931375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:19.082634 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8481-8b06-4645-9c06-9e12c0931375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:19.082930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8481-8b06-4645-9c06-9e12c0931375 admin admin] PATH_INFO: `/v3/projects/05549e93792e46068a3f8e2dda23265b/users/943cc71c206f48629f7e04e0d23ce2ba/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:19.097962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b8481-8b06-4645-9c06-9e12c0931375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05549e93792e46068a3f8e2dda23265b, user_id=943cc71c206f48629f7e04e0d23ce2ba, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:19.100690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b8481-8b06-4645-9c06-9e12c0931375 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:19.113616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1199/2401] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:19 2022] PUT /identity/v3/projects/05549e93792e46068a3f8e2dda23265b/users/943cc71c206f48629f7e04e0d23ce2ba/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:19.117404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda53283-48d3-48c3-881d-e557ffee4ad9 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:19.127534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda53283-48d3-48c3-881d-e557ffee4ad9 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:19.177840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:19.189097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2822-40d9-4ce5-ae84-2296fd877b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:19.189702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2822-40d9-4ce5-ae84-2296fd877b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:19.189980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2822-40d9-4ce5-ae84-2296fd877b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:19.190251 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2822-40d9-4ce5-ae84-2296fd877b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:19.190885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2822-40d9-4ce5-ae84-2296fd877b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:19.192522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2822-40d9-4ce5-ae84-2296fd877b36 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:19.203677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1203/2402] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:19.210626 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8481-8b06-4645-9c06-9e12c0931375 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:19.221935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8481-8b06-4645-9c06-9e12c0931375 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:19.226677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:19.227488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:19.227843 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:19.228187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] PATH_INFO: `/v3/projects/05549e93792e46068a3f8e2dda23265b/users/943cc71c206f48629f7e04e0d23ce2ba/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:19.230335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05549e93792e46068a3f8e2dda23265b, user_id=943cc71c206f48629f7e04e0d23ce2ba, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:19.233038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:19.243660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1200/2403] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:19 2022] PUT /identity/v3/projects/05549e93792e46068a3f8e2dda23265b/users/943cc71c206f48629f7e04e0d23ce2ba/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:19.309566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5820ca7-81fd-4cd9-870f-3f89abb9bdc6 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:19.309860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5820ca7-81fd-4cd9-870f-3f89abb9bdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:19.310157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5820ca7-81fd-4cd9-870f-3f89abb9bdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:19.338543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d5820ca7-81fd-4cd9-870f-3f89abb9bdc6 None None] MFA Rules not processed for user `943cc71c206f48629f7e04e0d23ce2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:19.339763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d5820ca7-81fd-4cd9-870f-3f89abb9bdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:19.406611 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1204/2404] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:19.417994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:19.419753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:19.442319 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:19.478737 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f55a-d3db-486d-9184-f1058c4a4109 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:19.482372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720d48d-9dfd-4804-9a50-25ea15dd1feb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:19.482995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720d48d-9dfd-4804-9a50-25ea15dd1feb service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:19.483278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720d48d-9dfd-4804-9a50-25ea15dd1feb service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:19.483552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720d48d-9dfd-4804-9a50-25ea15dd1feb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:19.492383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720d48d-9dfd-4804-9a50-25ea15dd1feb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:19.494286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720d48d-9dfd-4804-9a50-25ea15dd1feb service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:19.504036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1201/2405] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:19.588024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5820ca7-81fd-4cd9-870f-3f89abb9bdc6 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:19.589408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d5820ca7-81fd-4cd9-870f-3f89abb9bdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:19.601168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5820ca7-81fd-4cd9-870f-3f89abb9bdc6 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:19.604756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce0b93-969b-464d-9197-0f53ef809a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:19.605358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce0b93-969b-464d-9197-0f53ef809a6c service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:19.605640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce0b93-969b-464d-9197-0f53ef809a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:19.605912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce0b93-969b-464d-9197-0f53ef809a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:19.661493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce0b93-969b-464d-9197-0f53ef809a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:19.662852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce0b93-969b-464d-9197-0f53ef809a6c service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:19.672258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1205/2406] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:24.369191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720d48d-9dfd-4804-9a50-25ea15dd1feb service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:24.436718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720d48d-9dfd-4804-9a50-25ea15dd1feb service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:24.441499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:24.442121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:24.442397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:24.442668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] PATH_INFO: `/v3/users/f50977ff800b4c15b78da87a139ef3ee` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:24.443817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50977ff800b4c15b78da87a139ef3ee)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:24.445713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:24.493340 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50977ff800b4c15b78da87a139ef3ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:24.500376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50977ff800b4c15b78da87a139ef3ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:24.501027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50977ff800b4c15b78da87a139ef3ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:24.501633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50977ff800b4c15b78da87a139ef3ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:24.509826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50977ff800b4c15b78da87a139ef3ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:24.528121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1202/2407] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:24 2022] DELETE /identity/v3/users/f50977ff800b4c15b78da87a139ef3ee => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:24.532823 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc664e8-4531-4a71-8f4e-e05d527e0017 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:24.533299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc664e8-4531-4a71-8f4e-e05d527e0017 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:24.533542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc664e8-4531-4a71-8f4e-e05d527e0017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:24.553404 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6dc664e8-4531-4a71-8f4e-e05d527e0017 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:24.554597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6dc664e8-4531-4a71-8f4e-e05d527e0017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:24.615681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1206/2408] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:24.625062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:24.626519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:24.679360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:24.692940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac59d8-d0c6-4dfa-a706-7055ff34bf26 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:24.703281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e33a-a06f-45a8-80c9-a62f1bda6878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:24.703906 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e33a-a06f-45a8-80c9-a62f1bda6878 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:24.704212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e33a-a06f-45a8-80c9-a62f1bda6878 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:24.704516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e33a-a06f-45a8-80c9-a62f1bda6878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:24.715581 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415e33a-a06f-45a8-80c9-a62f1bda6878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:24.717052 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415e33a-a06f-45a8-80c9-a62f1bda6878 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:24.727526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1203/2409] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:25.020868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc664e8-4531-4a71-8f4e-e05d527e0017 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:25.022383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6dc664e8-4531-4a71-8f4e-e05d527e0017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:25.037259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc664e8-4531-4a71-8f4e-e05d527e0017 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:25.043410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:25.044130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:25.044517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:25.044863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] PATH_INFO: `/v3/projects/22dd25dc1f5c4041b366be4b892333a5` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:25.046086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dd25dc1f5c4041b366be4b892333a5)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:25.048052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:25.057779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:25.101696 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dd25dc1f5c4041b366be4b892333a5', 'request_id': 'req-ad0e5fea-e52a-421b-8ab0-a0c24a818102'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.112064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dd25dc1f5c4041b366be4b892333a5', 'request_id': 'req-ad0e5fea-e52a-421b-8ab0-a0c24a818102'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.119446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dd25dc1f5c4041b366be4b892333a5', 'request_id': 'req-ad0e5fea-e52a-421b-8ab0-a0c24a818102'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.129971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] The token cache is being invalidate because project 22dd25dc1f5c4041b366be4b892333a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:25.130256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.130919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.132428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1207/2410] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:25 2022] DELETE /identity/v3/projects/22dd25dc1f5c4041b366be4b892333a5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:25.136764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e33a-a06f-45a8-80c9-a62f1bda6878 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:25.146631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e33a-a06f-45a8-80c9-a62f1bda6878 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:25.209842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:25.210726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:25.211125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:25.211591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] PATH_INFO: `/v3/users/943cc71c206f48629f7e04e0d23ce2ba` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:25.212953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=943cc71c206f48629f7e04e0d23ce2ba)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:25.215194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:25.266844 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '943cc71c206f48629f7e04e0d23ce2ba'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.274624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '943cc71c206f48629f7e04e0d23ce2ba'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.275366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '943cc71c206f48629f7e04e0d23ce2ba'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.276136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '943cc71c206f48629f7e04e0d23ce2ba'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.284380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4312bd44-4998-4a82-bc3a-120f5d41b0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '943cc71c206f48629f7e04e0d23ce2ba'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.303785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1204/2411] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:25 2022] DELETE /identity/v3/users/943cc71c206f48629f7e04e0d23ce2ba => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:25.591788 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:25.593220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:25.606312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e5fea-e52a-421b-8ab0-a0c24a818102 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:25.664292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:25.664942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:25.665245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:25.665505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] PATH_INFO: `/v3/projects/05549e93792e46068a3f8e2dda23265b` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:25.666618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05549e93792e46068a3f8e2dda23265b)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:25.668568 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:25.712882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05549e93792e46068a3f8e2dda23265b', 'request_id': 'req-69a221d2-0f59-477c-8ea2-13d744ee2d50'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.720240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05549e93792e46068a3f8e2dda23265b', 'request_id': 'req-69a221d2-0f59-477c-8ea2-13d744ee2d50'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.726541 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05549e93792e46068a3f8e2dda23265b', 'request_id': 'req-69a221d2-0f59-477c-8ea2-13d744ee2d50'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.736649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] The token cache is being invalidate because project 05549e93792e46068a3f8e2dda23265b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:25.736933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.737608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-69a221d2-0f59-477c-8ea2-13d744ee2d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:25.739236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1208/2412] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:25 2022] DELETE /identity/v3/projects/05549e93792e46068a3f8e2dda23265b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:25.744597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d577ebf-38c6-4da5-83bd-d18eb52aa310 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:25.745062 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d577ebf-38c6-4da5-83bd-d18eb52aa310 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:25.745242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d577ebf-38c6-4da5-83bd-d18eb52aa310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:25.765158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8d577ebf-38c6-4da5-83bd-d18eb52aa310 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:25.766375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8d577ebf-38c6-4da5-83bd-d18eb52aa310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:25.824389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1205/2413] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:25.831996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105f24-d516-49be-bbea-c4f8baa3ebcc None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:25.832283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105f24-d516-49be-bbea-c4f8baa3ebcc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:25.832561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105f24-d516-49be-bbea-c4f8baa3ebcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:25.854758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ef105f24-d516-49be-bbea-c4f8baa3ebcc None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:25.856014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ef105f24-d516-49be-bbea-c4f8baa3ebcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:25.863897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1209/2414] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:25.868112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d577ebf-38c6-4da5-83bd-d18eb52aa310 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:25.874033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:25.878173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d577ebf-38c6-4da5-83bd-d18eb52aa310 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:25.885656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:25.886530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:25.887059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:25.887336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:25.887988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:25.888290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:25.889866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:25.902533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1206/2415] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:05:25.907142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105f24-d516-49be-bbea-c4f8baa3ebcc None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:25.916548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105f24-d516-49be-bbea-c4f8baa3ebcc None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:25.921135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec17e-6594-40a3-8fde-b76fe36dc333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:25.921737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec17e-6594-40a3-8fde-b76fe36dc333 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:25.922017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec17e-6594-40a3-8fde-b76fe36dc333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:25.922291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec17e-6594-40a3-8fde-b76fe36dc333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:25.922950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fec17e-6594-40a3-8fde-b76fe36dc333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:25.925115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fec17e-6594-40a3-8fde-b76fe36dc333 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:25.948508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:25.958597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1210/2416] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:25 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:25.963399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:25.975171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4d3d-c7ec-4084-a032-035a78a5d01e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:26.036491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19c042-5b5f-495a-913a-901a27a06e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:26.037094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19c042-5b5f-495a-913a-901a27a06e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.037377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19c042-5b5f-495a-913a-901a27a06e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.037650 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19c042-5b5f-495a-913a-901a27a06e2e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.038269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19c042-5b5f-495a-913a-901a27a06e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:26.040270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19c042-5b5f-495a-913a-901a27a06e2e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:26.060321 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1207/2417] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:25 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:26.064749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec17e-6594-40a3-8fde-b76fe36dc333 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:26.075414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec17e-6594-40a3-8fde-b76fe36dc333 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:26.080019 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a105b-b648-4f04-9fd4-cc09241bc6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:26.080616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a105b-b648-4f04-9fd4-cc09241bc6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.080898 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a105b-b648-4f04-9fd4-cc09241bc6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.081172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a105b-b648-4f04-9fd4-cc09241bc6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.081812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a105b-b648-4f04-9fd4-cc09241bc6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:26.083440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a105b-b648-4f04-9fd4-cc09241bc6d8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:26.094329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1211/2418] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:26.098292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19c042-5b5f-495a-913a-901a27a06e2e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:26.108499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19c042-5b5f-495a-913a-901a27a06e2e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:26.113139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a8081-7c36-4326-bd43-9ea53749d0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:26.113730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8081-7c36-4326-bd43-9ea53749d0dd admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.114011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8081-7c36-4326-bd43-9ea53749d0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.114285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8081-7c36-4326-bd43-9ea53749d0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.114975 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a8081-7c36-4326-bd43-9ea53749d0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:26.116549 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a8081-7c36-4326-bd43-9ea53749d0dd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:26.127155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1208/2419] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:05:26.131482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a105b-b648-4f04-9fd4-cc09241bc6d8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:26.141904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a105b-b648-4f04-9fd4-cc09241bc6d8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:26.146434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:26.147096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.147374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.147655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] PATH_INFO: `/v3/projects/eb2f9d6fe5b14cc7b805923bc02fd044/users/3d86ecb7373846afb122e437de43adc6/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.162189 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2f9d6fe5b14cc7b805923bc02fd044, user_id=3d86ecb7373846afb122e437de43adc6, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:26.164858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:26.177810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1212/2420] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:26 2022] PUT /identity/v3/projects/eb2f9d6fe5b14cc7b805923bc02fd044/users/3d86ecb7373846afb122e437de43adc6/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:26.189136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d4700-522a-4887-8bb4-dd91ae10f378 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.189425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d4700-522a-4887-8bb4-dd91ae10f378 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.189703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d4700-522a-4887-8bb4-dd91ae10f378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.218768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-503d4700-522a-4887-8bb4-dd91ae10f378 None None] MFA Rules not processed for user `3d86ecb7373846afb122e437de43adc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:26.219996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-503d4700-522a-4887-8bb4-dd91ae10f378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:26.289703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1209/2421] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:26.302022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:26.303776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:26.379281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4579b-d1fa-4ecc-b2c2-cbea6f6e040d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:26.384556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6f52a-7c6b-4417-8440-3e5a74774f60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:26.385577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6f52a-7c6b-4417-8440-3e5a74774f60 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.386064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6f52a-7c6b-4417-8440-3e5a74774f60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.388706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6f52a-7c6b-4417-8440-3e5a74774f60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.401029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6f52a-7c6b-4417-8440-3e5a74774f60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:26.402884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6f52a-7c6b-4417-8440-3e5a74774f60 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:26.412546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1213/2422] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:26.434212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d4700-522a-4887-8bb4-dd91ae10f378 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:26.504087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d4700-522a-4887-8bb4-dd91ae10f378 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:26.509301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:26.509933 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.510215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.510488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] PATH_INFO: `/v3/users/3d86ecb7373846afb122e437de43adc6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.512165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d86ecb7373846afb122e437de43adc6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:26.514145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:26.515835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:26.592960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d86ecb7373846afb122e437de43adc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:26.601282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d86ecb7373846afb122e437de43adc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:26.602185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d86ecb7373846afb122e437de43adc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:26.603419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d86ecb7373846afb122e437de43adc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:26.611471 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d86ecb7373846afb122e437de43adc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:26.637478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1210/2423] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:26 2022] DELETE /identity/v3/users/3d86ecb7373846afb122e437de43adc6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:26.643046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe85c17-edb7-4c82-89d6-23a0be1f8768 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.643466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe85c17-edb7-4c82-89d6-23a0be1f8768 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.643653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe85c17-edb7-4c82-89d6-23a0be1f8768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.664702 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ebe85c17-edb7-4c82-89d6-23a0be1f8768 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:26.666532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ebe85c17-edb7-4c82-89d6-23a0be1f8768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:26.726560 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1214/2424] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:26.737122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:26.738515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:26.797588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c50a6-764c-48b7-83d6-ac1c95c02ca8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:26.801555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b33c8-cf06-408f-9e63-1c5d3b0d072a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:26.802157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b33c8-cf06-408f-9e63-1c5d3b0d072a service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:26.802437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b33c8-cf06-408f-9e63-1c5d3b0d072a service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:26.802714 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b33c8-cf06-408f-9e63-1c5d3b0d072a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:26.812348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b33c8-cf06-408f-9e63-1c5d3b0d072a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:26.813768 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b33c8-cf06-408f-9e63-1c5d3b0d072a service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:26.823173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1211/2425] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:05:27.081618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe85c17-edb7-4c82-89d6-23a0be1f8768 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:27.083059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ebe85c17-edb7-4c82-89d6-23a0be1f8768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:27.095542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe85c17-edb7-4c82-89d6-23a0be1f8768 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:27.106079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:27.106740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.107051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.107318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] PATH_INFO: `/v3/projects/eb2f9d6fe5b14cc7b805923bc02fd044` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.108584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb2f9d6fe5b14cc7b805923bc02fd044)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:27.110484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:27.156534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb2f9d6fe5b14cc7b805923bc02fd044', 'request_id': 'req-70335276-c0e8-47a1-82c0-28f90bd3a4d2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:27.164136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb2f9d6fe5b14cc7b805923bc02fd044', 'request_id': 'req-70335276-c0e8-47a1-82c0-28f90bd3a4d2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:27.171292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb2f9d6fe5b14cc7b805923bc02fd044', 'request_id': 'req-70335276-c0e8-47a1-82c0-28f90bd3a4d2'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:27.172650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:27.184416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] The token cache is being invalidate because project eb2f9d6fe5b14cc7b805923bc02fd044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:27.184707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:27.185424 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-70335276-c0e8-47a1-82c0-28f90bd3a4d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:27.187110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1215/2426] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:27 2022] DELETE /identity/v3/projects/eb2f9d6fe5b14cc7b805923bc02fd044 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:27.192116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c57e-3702-4bcc-bbe8-f1924b40e4b6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.192410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c57e-3702-4bcc-bbe8-f1924b40e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.192713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c57e-3702-4bcc-bbe8-f1924b40e4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.213297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a9b9c57e-3702-4bcc-bbe8-f1924b40e4b6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:27.214508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a9b9c57e-3702-4bcc-bbe8-f1924b40e4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:27.273853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1212/2427] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:27.282040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03597a6-c1a7-4149-a2e8-58567ed27563 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.282326 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03597a6-c1a7-4149-a2e8-58567ed27563 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.282604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03597a6-c1a7-4149-a2e8-58567ed27563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.302633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e03597a6-c1a7-4149-a2e8-58567ed27563 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:27.303863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e03597a6-c1a7-4149-a2e8-58567ed27563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:27.312085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1216/2428] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:27.316189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9c57e-3702-4bcc-bbe8-f1924b40e4b6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:27.325704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9c57e-3702-4bcc-bbe8-f1924b40e4b6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:27.330268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:27.330861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.331227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.331557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.332080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:27.332382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:27.334016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:27.345792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1213/2429] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:05:27.349982 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03597a6-c1a7-4149-a2e8-58567ed27563 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:27.359924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03597a6-c1a7-4149-a2e8-58567ed27563 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:27.364453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f46c-b21b-4400-8f7c-b016d4f46272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:27.365062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f46c-b21b-4400-8f7c-b016d4f46272 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.365338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f46c-b21b-4400-8f7c-b016d4f46272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.365609 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f46c-b21b-4400-8f7c-b016d4f46272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.366216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15f46c-b21b-4400-8f7c-b016d4f46272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:27.368139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15f46c-b21b-4400-8f7c-b016d4f46272 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:27.397048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1217/2430] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:27 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:27.406196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:27.416475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4f31c-ebcd-42c2-843d-000fb604216e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:27.421152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:27.477155 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be5f7b-6073-4b60-ac13-af10ca727ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:27.477762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5f7b-6073-4b60-ac13-af10ca727ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.478039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5f7b-6073-4b60-ac13-af10ca727ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.478309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5f7b-6073-4b60-ac13-af10ca727ddc admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.479028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be5f7b-6073-4b60-ac13-af10ca727ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:27.480950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be5f7b-6073-4b60-ac13-af10ca727ddc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:27.502244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1214/2431] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:27 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:27.506421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f46c-b21b-4400-8f7c-b016d4f46272 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:27.517107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f46c-b21b-4400-8f7c-b016d4f46272 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:27.521621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d816af-8d16-4357-9d56-07ccf2a97663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:27.522218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d816af-8d16-4357-9d56-07ccf2a97663 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.522493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d816af-8d16-4357-9d56-07ccf2a97663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.522760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d816af-8d16-4357-9d56-07ccf2a97663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.523423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d816af-8d16-4357-9d56-07ccf2a97663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:27.525651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d816af-8d16-4357-9d56-07ccf2a97663 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:27.536711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1218/2432] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:27.540087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be5f7b-6073-4b60-ac13-af10ca727ddc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:27.550748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be5f7b-6073-4b60-ac13-af10ca727ddc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:27.555481 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db6f6-8865-4497-9e66-cce96cf46c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:27.556078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db6f6-8865-4497-9e66-cce96cf46c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.556363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db6f6-8865-4497-9e66-cce96cf46c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.556662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db6f6-8865-4497-9e66-cce96cf46c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.557357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247db6f6-8865-4497-9e66-cce96cf46c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:27.558966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247db6f6-8865-4497-9e66-cce96cf46c75 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:27.570245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1215/2433] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:27.575062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d816af-8d16-4357-9d56-07ccf2a97663 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:27.584738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d816af-8d16-4357-9d56-07ccf2a97663 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:27.590977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:27.591700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.592048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.594092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] PATH_INFO: `/v3/projects/0223b57233664d0785c96ec98335e307/users/0243f437de8b439984375303ff49f9e3/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.610208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0223b57233664d0785c96ec98335e307, user_id=0243f437de8b439984375303ff49f9e3, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:27.613115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:27.631641 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1219/2434] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:27 2022] PUT /identity/v3/projects/0223b57233664d0785c96ec98335e307/users/0243f437de8b439984375303ff49f9e3/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:27.637216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da083a4e-0928-483b-b72f-57d252294830 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.637523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da083a4e-0928-483b-b72f-57d252294830 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.637776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da083a4e-0928-483b-b72f-57d252294830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.659332 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-da083a4e-0928-483b-b72f-57d252294830 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:27.660463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-da083a4e-0928-483b-b72f-57d252294830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:27.733541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1216/2435] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:27.743937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:27.745685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:27.807056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb37b2e-b11c-49c4-9072-5bd4a234aea7 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:27.810723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:27.811375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:27.811653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:27.811925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:27.822024 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:27.823562 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:27.833215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1220/2436] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:28.475780 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f4d28-d6a3-42a8-abb8-29870aa92955 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:28.476305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f4d28-d6a3-42a8-abb8-29870aa92955 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:28.476305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f4d28-d6a3-42a8-abb8-29870aa92955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:28.503739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-384f4d28-d6a3-42a8-abb8-29870aa92955 None None] MFA Rules not processed for user `0243f437de8b439984375303ff49f9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:28.504940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-384f4d28-d6a3-42a8-abb8-29870aa92955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:28.573273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1217/2437] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:28.583858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:28.585263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:28.643252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca3ee-7faf-4650-8cd0-428e5ea3d999 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:28.648415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd60d-dc50-4c81-b49f-22e350e25d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:28.649093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd60d-dc50-4c81-b49f-22e350e25d22 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:28.650101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd60d-dc50-4c81-b49f-22e350e25d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:28.650429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd60d-dc50-4c81-b49f-22e350e25d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:28.659238 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd60d-dc50-4c81-b49f-22e350e25d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:28.660925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd60d-dc50-4c81-b49f-22e350e25d22 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:28.672188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1221/2438] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:30.075610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324e54a-b0af-4c1b-ace5-813f7dd7b4f8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:30.076074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324e54a-b0af-4c1b-ace5-813f7dd7b4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:30.076196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324e54a-b0af-4c1b-ace5-813f7dd7b4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:30.095692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e324e54a-b0af-4c1b-ace5-813f7dd7b4f8 None None] MFA Rules not processed for user `0243f437de8b439984375303ff49f9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:30.096889 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e324e54a-b0af-4c1b-ace5-813f7dd7b4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:30.104655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1218/2439] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:38.667989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=2,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:40.090337 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:42.097458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdf412-e893-493f-8e18-a6a118fe352f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:42.097846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdf412-e893-493f-8e18-a6a118fe352f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:42.097953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdf412-e893-493f-8e18-a6a118fe352f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:42.125815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-f0fdf412-e893-493f-8e18-a6a118fe352f None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:42.127076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f0fdf412-e893-493f-8e18-a6a118fe352f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:42.192231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1222/2440] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:05:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:42.241816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324e54a-b0af-4c1b-ace5-813f7dd7b4f8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:42.243759 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e324e54a-b0af-4c1b-ace5-813f7dd7b4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:42.323986 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324e54a-b0af-4c1b-ace5-813f7dd7b4f8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:42.329126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:42.329950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:42.330296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:42.330632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:42.339187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:42.341152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:42.352703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1219/2441] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:05:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:05:47.217376 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdf412-e893-493f-8e18-a6a118fe352f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:47.228270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdf412-e893-493f-8e18-a6a118fe352f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:47.232749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:47.233655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:47.233940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:47.234215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:47.236034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:47.250286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:47.253029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:47.266902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1223/2442] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:50.409749 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:50.411904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:50.428895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5264ba-a8b3-444d-b275-fb05668e001f service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:50.437249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:50.438255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:50.438741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:50.439261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:50.441301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:50.465390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:50.467363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:50.482119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1220/2443] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:52.064318 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:52.071662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:52.157249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:52.158968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:52.183433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e83bda-8dac-4967-a9de-fba01fc83a90 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:52.194194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:52.194943 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:52.195295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:52.196838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] PATH_INFO: `/v3/users/0c0f1d116c004694b33b19d988ac4959` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:52.198736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0f1d116c004694b33b19d988ac4959)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:52.209912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:52.297935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36870783-def7-4177-a5d0-b5d771114f6c service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:52.320933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:52.323449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:52.326203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:52.335563 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:52.343185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:52.358571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0f1d116c004694b33b19d988ac4959'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:52.381016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:52.381899 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0f1d116c004694b33b19d988ac4959'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:52.385010 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:52.388339 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0f1d116c004694b33b19d988ac4959'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:52.392447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0f1d116c004694b33b19d988ac4959'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:52.423327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d696b9e-730c-41e2-bb2a-f1fdd1f44290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0f1d116c004694b33b19d988ac4959'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:52.439882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1224/2444] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:52.472103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1221/2445] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:52 2022] DELETE /identity/v3/users/0c0f1d116c004694b33b19d988ac4959 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:53.113576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:53.126220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63234d44-68df-47fe-801b-fc8c748b0ace service placement] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:53.224480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:53.225884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.227096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.227612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] PATH_INFO: `/v3/projects/79aee39298624f99b33b9808fe5030f8` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.228941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79aee39298624f99b33b9808fe5030f8)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:53.230947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:53.265758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:53.297097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79aee39298624f99b33b9808fe5030f8', 'request_id': 'req-7948d741-4126-4cc6-bd0a-c2185de25f35'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:53.313226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79aee39298624f99b33b9808fe5030f8', 'request_id': 'req-7948d741-4126-4cc6-bd0a-c2185de25f35'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:53.328715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79aee39298624f99b33b9808fe5030f8', 'request_id': 'req-7948d741-4126-4cc6-bd0a-c2185de25f35'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:53.343188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] The token cache is being invalidate because project 79aee39298624f99b33b9808fe5030f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:53.343600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:53.344335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7948d741-4126-4cc6-bd0a-c2185de25f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:53.349020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1225/2446] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:53 2022] DELETE /identity/v3/projects/79aee39298624f99b33b9808fe5030f8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:53.354957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082543a-4b27-4e2e-88da-bb08ba3c5af8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.355503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082543a-4b27-4e2e-88da-bb08ba3c5af8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.356891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082543a-4b27-4e2e-88da-bb08ba3c5af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.385915 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3082543a-4b27-4e2e-88da-bb08ba3c5af8 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:53.387432 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3082543a-4b27-4e2e-88da-bb08ba3c5af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:53.450208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:53.477398 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1222/2447] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:53.484782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d16df-471b-464b-b2ad-bc20c57358f2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.485172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d16df-471b-464b-b2ad-bc20c57358f2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.485976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d16df-471b-464b-b2ad-bc20c57358f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.511792 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-740d16df-471b-464b-b2ad-bc20c57358f2 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:53.513888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-740d16df-471b-464b-b2ad-bc20c57358f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:53.527129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1226/2448] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:53.527652 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082543a-4b27-4e2e-88da-bb08ba3c5af8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:53.546445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082543a-4b27-4e2e-88da-bb08ba3c5af8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:53.554114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:53.554792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.555106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.555665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.557154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:53.557956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:53.560335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:53.580479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1223/2449] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:05:53.584594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d16df-471b-464b-b2ad-bc20c57358f2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:53.601040 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d16df-471b-464b-b2ad-bc20c57358f2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:53.609236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b245e99-244b-4af6-a21d-25f430c989c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:53.610872 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b245e99-244b-4af6-a21d-25f430c989c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.611867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b245e99-244b-4af6-a21d-25f430c989c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.612299 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b245e99-244b-4af6-a21d-25f430c989c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.615339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b245e99-244b-4af6-a21d-25f430c989c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:53.617493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b245e99-244b-4af6-a21d-25f430c989c7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:53.667951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1227/2450] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:53 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:53.669560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:53.682226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb8c31-05a4-4a2b-b8f9-df1ecf8d2fa9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:53.789283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946841a-b571-4432-b053-155a44ff3c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:53.790433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946841a-b571-4432-b053-155a44ff3c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.791318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946841a-b571-4432-b053-155a44ff3c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.791318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946841a-b571-4432-b053-155a44ff3c39 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.793663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946841a-b571-4432-b053-155a44ff3c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:53.799783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946841a-b571-4432-b053-155a44ff3c39 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:53.852316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1224/2451] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:53 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:53.856394 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b245e99-244b-4af6-a21d-25f430c989c7 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:53.869879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b245e99-244b-4af6-a21d-25f430c989c7 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:53.885974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25487a8-eab8-4230-b1a3-6d4922736794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:53.886604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25487a8-eab8-4230-b1a3-6d4922736794 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.886915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25487a8-eab8-4230-b1a3-6d4922736794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.887183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25487a8-eab8-4230-b1a3-6d4922736794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.887837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25487a8-eab8-4230-b1a3-6d4922736794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:53.889948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25487a8-eab8-4230-b1a3-6d4922736794 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:53.912853 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1228/2452] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:53.917367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946841a-b571-4432-b053-155a44ff3c39 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:53.930140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946841a-b571-4432-b053-155a44ff3c39 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:53.935640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7117937-4be1-4703-b209-7e26853990fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:53.936423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117937-4be1-4703-b209-7e26853990fc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.936804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117937-4be1-4703-b209-7e26853990fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.937175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117937-4be1-4703-b209-7e26853990fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.938051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7117937-4be1-4703-b209-7e26853990fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:53.939810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7117937-4be1-4703-b209-7e26853990fc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:53.961583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1225/2453] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:53.962274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25487a8-eab8-4230-b1a3-6d4922736794 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:53.972770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25487a8-eab8-4230-b1a3-6d4922736794 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:53.991382 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e132c-83b9-49c2-b155-a080cd32c067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:53.991904 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e132c-83b9-49c2-b155-a080cd32c067 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.992842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e132c-83b9-49c2-b155-a080cd32c067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.993226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e132c-83b9-49c2-b155-a080cd32c067 admin admin] PATH_INFO: `/v3/projects/37719c37cc4c463da610145296ef811f/users/454183ac8f4348228ec4643336c66400/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.021406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e132c-83b9-49c2-b155-a080cd32c067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37719c37cc4c463da610145296ef811f, user_id=454183ac8f4348228ec4643336c66400, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:54.024821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e132c-83b9-49c2-b155-a080cd32c067 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:54.042291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1229/2454] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:53 2022] PUT /identity/v3/projects/37719c37cc4c463da610145296ef811f/users/454183ac8f4348228ec4643336c66400/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:54.057686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba8772-1908-478a-bba2-52e55ae5becd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:54.058115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba8772-1908-478a-bba2-52e55ae5becd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:54.058552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba8772-1908-478a-bba2-52e55ae5becd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.104604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-33ba8772-1908-478a-bba2-52e55ae5becd None None] MFA Rules not processed for user `454183ac8f4348228ec4643336c66400`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:54.106663 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-33ba8772-1908-478a-bba2-52e55ae5becd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:54.235609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1226/2455] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:54.249270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e132c-83b9-49c2-b155-a080cd32c067 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:54.349894 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e132c-83b9-49c2-b155-a080cd32c067 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:54.356124 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c2bdd-e049-44de-bca1-00b856f9b20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:54.356740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c2bdd-e049-44de-bca1-00b856f9b20c admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:54.357021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c2bdd-e049-44de-bca1-00b856f9b20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:54.357294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c2bdd-e049-44de-bca1-00b856f9b20c admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.358039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c2bdd-e049-44de-bca1-00b856f9b20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:54.360396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c2bdd-e049-44de-bca1-00b856f9b20c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:54.399092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1230/2456] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:54 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:54.400785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba8772-1908-478a-bba2-52e55ae5becd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:54.419609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba8772-1908-478a-bba2-52e55ae5becd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:54.474667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:54.495202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67ad7e-b86c-4d48-98dd-3cccfc96cc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:54.495840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67ad7e-b86c-4d48-98dd-3cccfc96cc48 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:54.496673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67ad7e-b86c-4d48-98dd-3cccfc96cc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:54.496673 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67ad7e-b86c-4d48-98dd-3cccfc96cc48 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.497323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67ad7e-b86c-4d48-98dd-3cccfc96cc48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:54.500892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67ad7e-b86c-4d48-98dd-3cccfc96cc48 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:54.522236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1227/2457] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:54 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:54.526342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c2bdd-e049-44de-bca1-00b856f9b20c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:54.537484 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c2bdd-e049-44de-bca1-00b856f9b20c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:54.542628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1d12a-d453-4c86-bdb0-2c8c58ae762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:54.543274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1d12a-d453-4c86-bdb0-2c8c58ae762c admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:54.543594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1d12a-d453-4c86-bdb0-2c8c58ae762c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:54.543832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1d12a-d453-4c86-bdb0-2c8c58ae762c admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.544584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1d12a-d453-4c86-bdb0-2c8c58ae762c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:54.546462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1d12a-d453-4c86-bdb0-2c8c58ae762c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:54.558436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1231/2458] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:54.563096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67ad7e-b86c-4d48-98dd-3cccfc96cc48 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:54.573551 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67ad7e-b86c-4d48-98dd-3cccfc96cc48 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:54.578465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042935db-bd1e-442f-a39b-967018450cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:54.579176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042935db-bd1e-442f-a39b-967018450cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:54.579460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042935db-bd1e-442f-a39b-967018450cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:54.579733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042935db-bd1e-442f-a39b-967018450cc0 admin admin] PATH_INFO: `/v3/projects/ce2502e78c064b1b8e343ed0b52b20e4/users/ed064dac047a428eae9674c194604185/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.599866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042935db-bd1e-442f-a39b-967018450cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2502e78c064b1b8e343ed0b52b20e4, user_id=ed064dac047a428eae9674c194604185, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:54.603378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042935db-bd1e-442f-a39b-967018450cc0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:54.623284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1228/2459] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:54 2022] PUT /identity/v3/projects/ce2502e78c064b1b8e343ed0b52b20e4/users/ed064dac047a428eae9674c194604185/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:54.627298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1d12a-d453-4c86-bdb0-2c8c58ae762c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:54.638999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1d12a-d453-4c86-bdb0-2c8c58ae762c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:54.665029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:54.720674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324d85c-1326-4fe0-a01c-8a9200db08dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:54.721390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324d85c-1326-4fe0-a01c-8a9200db08dc admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:54.721737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324d85c-1326-4fe0-a01c-8a9200db08dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:54.722072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324d85c-1326-4fe0-a01c-8a9200db08dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.722869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324d85c-1326-4fe0-a01c-8a9200db08dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:54.724450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324d85c-1326-4fe0-a01c-8a9200db08dc admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:54.738227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1232/2460] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:54.744014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042935db-bd1e-442f-a39b-967018450cc0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:54.755136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042935db-bd1e-442f-a39b-967018450cc0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:54.760145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:54.760902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:54.761254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:54.761605 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] PATH_INFO: `/v3/projects/ce2502e78c064b1b8e343ed0b52b20e4/users/ed064dac047a428eae9674c194604185/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.763956 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2502e78c064b1b8e343ed0b52b20e4, user_id=ed064dac047a428eae9674c194604185, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:54.766808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:54.778770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1229/2461] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:54 2022] PUT /identity/v3/projects/ce2502e78c064b1b8e343ed0b52b20e4/users/ed064dac047a428eae9674c194604185/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:54.789622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d38e5-d460-4150-9451-303ef90dbd05 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:54.789952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d38e5-d460-4150-9451-303ef90dbd05 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:54.790423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d38e5-d460-4150-9451-303ef90dbd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:54.958185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bb2d38e5-d460-4150-9451-303ef90dbd05 None None] MFA Rules not processed for user `ed064dac047a428eae9674c194604185`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:54.959470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bb2d38e5-d460-4150-9451-303ef90dbd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:55.046946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1233/2462] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:55.059721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:55.062502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:55.155115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802180d5-7591-4dfa-a844-72324b4d8784 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:55.161384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b38d-3c76-4028-bc52-b58c35f90fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:55.162090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b38d-3c76-4028-bc52-b58c35f90fba service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:55.162440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b38d-3c76-4028-bc52-b58c35f90fba service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:55.162793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b38d-3c76-4028-bc52-b58c35f90fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:55.179147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b38d-3c76-4028-bc52-b58c35f90fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:55.182249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b38d-3c76-4028-bc52-b58c35f90fba service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:55.195447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1230/2463] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:55.761487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d38e5-d460-4150-9451-303ef90dbd05 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:55.763582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bb2d38e5-d460-4150-9451-303ef90dbd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:55.846314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b38d-3c76-4028-bc52-b58c35f90fba service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:55.852925 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d38e5-d460-4150-9451-303ef90dbd05 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:55.867426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:55.868004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:55.868282 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:55.868441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b38d-3c76-4028-bc52-b58c35f90fba service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:55.868556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] PATH_INFO: `/v3/users/2f7735444ca8431ea7906070374dbe67` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:55.869776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7735444ca8431ea7906070374dbe67)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:55.871819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:55.872854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:55.873661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:55.874374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:55.874374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] PATH_INFO: `/v3/users/454183ac8f4348228ec4643336c66400` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:55.875704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=454183ac8f4348228ec4643336c66400)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:55.878275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:55.926824 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:55.929229 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7735444ca8431ea7906070374dbe67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.929892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454183ac8f4348228ec4643336c66400'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.941629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454183ac8f4348228ec4643336c66400'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.942735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454183ac8f4348228ec4643336c66400'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.943591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454183ac8f4348228ec4643336c66400'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.944987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7735444ca8431ea7906070374dbe67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.946265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7735444ca8431ea7906070374dbe67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.946880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7735444ca8431ea7906070374dbe67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.955183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8fa87674-f687-4c17-a3ed-f36de20698c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454183ac8f4348228ec4643336c66400'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.960406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7735444ca8431ea7906070374dbe67'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:55.983362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1231/2464] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:55 2022] DELETE /identity/v3/users/454183ac8f4348228ec4643336c66400 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:55.988118 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b386a2b-9c80-41e4-9145-ebf5603c246e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:55.988577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b386a2b-9c80-41e4-9145-ebf5603c246e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:55.989029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b386a2b-9c80-41e4-9145-ebf5603c246e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:55.991284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1234/2465] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:55 2022] DELETE /identity/v3/users/2f7735444ca8431ea7906070374dbe67 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:56.012734 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7b386a2b-9c80-41e4-9145-ebf5603c246e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:56.014281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7b386a2b-9c80-41e4-9145-ebf5603c246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.019913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:56.088636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1232/2466] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:56.099275 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.100684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.176059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c772db-44cf-43b6-b3e4-a5faab60877d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.177148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b386a2b-9c80-41e4-9145-ebf5603c246e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.179077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7b386a2b-9c80-41e4-9145-ebf5603c246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.185371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a307ac8-4d1b-405d-95d7-bf2774aeae2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.186071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307ac8-4d1b-405d-95d7-bf2774aeae2e service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.186418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307ac8-4d1b-405d-95d7-bf2774aeae2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.186758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307ac8-4d1b-405d-95d7-bf2774aeae2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.193313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b386a2b-9c80-41e4-9145-ebf5603c246e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.199345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a307ac8-4d1b-405d-95d7-bf2774aeae2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.200787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a307ac8-4d1b-405d-95d7-bf2774aeae2e service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.202835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.203630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.204007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.204378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] PATH_INFO: `/v3/projects/de3e893bfb574016a6e5e020f96490c5` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.206969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de3e893bfb574016a6e5e020f96490c5)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.209276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.212949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1235/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:56.271949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de3e893bfb574016a6e5e020f96490c5', 'request_id': 'req-00762955-14dd-4117-9a98-1e1bab2f3dc7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.287229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de3e893bfb574016a6e5e020f96490c5', 'request_id': 'req-00762955-14dd-4117-9a98-1e1bab2f3dc7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.299023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de3e893bfb574016a6e5e020f96490c5', 'request_id': 'req-00762955-14dd-4117-9a98-1e1bab2f3dc7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.313310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] The token cache is being invalidate because project de3e893bfb574016a6e5e020f96490c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:56.313693 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.314517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00762955-14dd-4117-9a98-1e1bab2f3dc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.316672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1233/2468] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:56 2022] DELETE /identity/v3/projects/de3e893bfb574016a6e5e020f96490c5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:56.322650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a489-c4b9-419c-9107-905dfb12f95b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.322994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a489-c4b9-419c-9107-905dfb12f95b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.323285 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056a489-c4b9-419c-9107-905dfb12f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.344169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-7056a489-c4b9-419c-9107-905dfb12f95b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:56.345701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7056a489-c4b9-419c-9107-905dfb12f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.425108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1236/2469] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:56.433662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333e28e-03f6-402b-9ce7-4d4ddfc02252 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.433947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333e28e-03f6-402b-9ce7-4d4ddfc02252 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.434216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333e28e-03f6-402b-9ce7-4d4ddfc02252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.454779 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b333e28e-03f6-402b-9ce7-4d4ddfc02252 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:56.456020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b333e28e-03f6-402b-9ce7-4d4ddfc02252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.463878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1234/2470] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:56.468216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056a489-c4b9-419c-9107-905dfb12f95b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.477962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056a489-c4b9-419c-9107-905dfb12f95b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.482680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.483351 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.483634 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.484291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.485420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:56.486136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.487878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.500837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1237/2471] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:05:56.505540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333e28e-03f6-402b-9ce7-4d4ddfc02252 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.513586 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.515529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.517397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333e28e-03f6-402b-9ce7-4d4ddfc02252 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.522139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7b973-f38d-41a7-80fa-0d2db3bf6157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.522977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7b973-f38d-41a7-80fa-0d2db3bf6157 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.523480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7b973-f38d-41a7-80fa-0d2db3bf6157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.523832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7b973-f38d-41a7-80fa-0d2db3bf6157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.524543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7b973-f38d-41a7-80fa-0d2db3bf6157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.526809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7b973-f38d-41a7-80fa-0d2db3bf6157 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.529180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae1b6b-4603-4874-be08-a055ad37f52d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.533912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.534606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.535048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.535414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] PATH_INFO: `/v3/projects/37719c37cc4c463da610145296ef811f` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.536575 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=37719c37cc4c463da610145296ef811f)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.538588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.566200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1235/2472] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:56 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:56.571171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7b973-f38d-41a7-80fa-0d2db3bf6157 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.595624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:56.597242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37719c37cc4c463da610145296ef811f', 'request_id': 'req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.622452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37719c37cc4c463da610145296ef811f', 'request_id': 'req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.630377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37719c37cc4c463da610145296ef811f', 'request_id': 'req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.644475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] The token cache is being invalidate because project 37719c37cc4c463da610145296ef811f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:56.644854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.645591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.647358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1238/2473] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:56 2022] DELETE /identity/v3/projects/37719c37cc4c463da610145296ef811f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:56.651513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.653612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.662174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7b973-f38d-41a7-80fa-0d2db3bf6157 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.674601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:56.733876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c70e12-d1ba-4b5f-880b-052ce32ce8ba admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.734880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.735544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.735826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.736163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.736746 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.738751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.739469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.740294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.740675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.741053 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] PATH_INFO: `/v3/users/ed064dac047a428eae9674c194604185` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.742964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed064dac047a428eae9674c194604185)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.745021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.767744 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1236/2474] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:56 2022] POST /identity/v3/users => generated 285 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:56.771633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.774011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.796558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9295fe-1ed7-4d28-bb6e-be6ef579f324 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.804194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.805445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.805734 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.806016 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.806669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.808513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.810843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed064dac047a428eae9674c194604185'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.818486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed064dac047a428eae9674c194604185'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.819222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed064dac047a428eae9674c194604185'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.819839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed064dac047a428eae9674c194604185'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.822932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1237/2475] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:56.827159 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.829938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:56.835157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed064dac047a428eae9674c194604185'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:56.846268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29450989-3de2-4089-84c7-b43a49102951 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.850479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61229427-b751-407d-813c-a909805bbd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.851117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229427-b751-407d-813c-a909805bbd6b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.851370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229427-b751-407d-813c-a909805bbd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.852199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61229427-b751-407d-813c-a909805bbd6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.852830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61229427-b751-407d-813c-a909805bbd6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.854399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61229427-b751-407d-813c-a909805bbd6b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.862624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1239/2476] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:56 2022] DELETE /identity/v3/users/ed064dac047a428eae9674c194604185 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:56.865249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1238/2477] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:56.871032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:56.946341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527f955-cea9-4e69-873c-62fcfe42e48a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:56.953626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:56.954812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29291-dd5e-4ae9-823f-b273e6fdad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:56.955569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29291-dd5e-4ae9-823f-b273e6fdad7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:56.956003 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29291-dd5e-4ae9-823f-b273e6fdad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:56.956452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29291-dd5e-4ae9-823f-b273e6fdad7b admin admin] PATH_INFO: `/v3/projects/bd1c0d6ad536480fb7d50ae3dd7571b2/users/0a29181eff26432195bde123afe9f556/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:56.973031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee29291-dd5e-4ae9-823f-b273e6fdad7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1c0d6ad536480fb7d50ae3dd7571b2, user_id=0a29181eff26432195bde123afe9f556, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:56.975770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee29291-dd5e-4ae9-823f-b273e6fdad7b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:56.989055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1240/2478] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:56 2022] PUT /identity/v3/projects/bd1c0d6ad536480fb7d50ae3dd7571b2/users/0a29181eff26432195bde123afe9f556/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:57.000937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853865-9f84-4f11-8b93-78f1a2eef675 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.001812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853865-9f84-4f11-8b93-78f1a2eef675 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.002491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853865-9f84-4f11-8b93-78f1a2eef675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.043101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-00853865-9f84-4f11-8b93-78f1a2eef675 None None] MFA Rules not processed for user `0a29181eff26432195bde123afe9f556`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:57.043101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-00853865-9f84-4f11-8b93-78f1a2eef675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:57.114341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1239/2479] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:57.118700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29291-dd5e-4ae9-823f-b273e6fdad7b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.155506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00853865-9f84-4f11-8b93-78f1a2eef675 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.156982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-00853865-9f84-4f11-8b93-78f1a2eef675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:57.201725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29291-dd5e-4ae9-823f-b273e6fdad7b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.207437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab293-777e-463d-96f9-583005570b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.208234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab293-777e-463d-96f9-583005570b81 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.208739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab293-777e-463d-96f9-583005570b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.209083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab293-777e-463d-96f9-583005570b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.210004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ab293-777e-463d-96f9-583005570b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.213720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ab293-777e-463d-96f9-583005570b81 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.237453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00853865-9f84-4f11-8b93-78f1a2eef675 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.241429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1241/2480] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:57.243510 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.244150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.244665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.244790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] PATH_INFO: `/v3/projects/ce2502e78c064b1b8e343ed0b52b20e4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.245477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab293-777e-463d-96f9-583005570b81 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.245832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2502e78c064b1b8e343ed0b52b20e4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.247789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.260248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:57.298767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2502e78c064b1b8e343ed0b52b20e4', 'request_id': 'req-8543d708-7567-4e48-bbe9-e278f42ae0a7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:57.307408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2502e78c064b1b8e343ed0b52b20e4', 'request_id': 'req-8543d708-7567-4e48-bbe9-e278f42ae0a7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:57.308594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab293-777e-463d-96f9-583005570b81 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.314017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2502e78c064b1b8e343ed0b52b20e4', 'request_id': 'req-8543d708-7567-4e48-bbe9-e278f42ae0a7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:57.324907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] The token cache is being invalidate because project ce2502e78c064b1b8e343ed0b52b20e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:05:57.325327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:57.326063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8543d708-7567-4e48-bbe9-e278f42ae0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:57.327740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1240/2481] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:57 2022] DELETE /identity/v3/projects/ce2502e78c064b1b8e343ed0b52b20e4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:57.332355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3168f17-b119-4f67-b8eb-c713bde541e1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.332796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3168f17-b119-4f67-b8eb-c713bde541e1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.333080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3168f17-b119-4f67-b8eb-c713bde541e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.354083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b3168f17-b119-4f67-b8eb-c713bde541e1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:57.355307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b3168f17-b119-4f67-b8eb-c713bde541e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:57.366058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.366656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.366977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.367272 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.367880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.369789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.389101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:57.395075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1242/2482] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:57.399815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.404695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:57.416617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e03b5-2d86-443c-8b68-877a6e88d5cd admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.421198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da32712-4ac8-4f0e-925b-cdbcdec95157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.421791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da32712-4ac8-4f0e-925b-cdbcdec95157 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.422068 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da32712-4ac8-4f0e-925b-cdbcdec95157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.422336 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da32712-4ac8-4f0e-925b-cdbcdec95157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.423000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da32712-4ac8-4f0e-925b-cdbcdec95157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.424857 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da32712-4ac8-4f0e-925b-cdbcdec95157 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.427176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1241/2483] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:57.435147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1243/2484] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:05:57.436004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f85634-b4f3-408c-9e2a-699efe739f4a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.436293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f85634-b4f3-408c-9e2a-699efe739f4a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.436566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f85634-b4f3-408c-9e2a-699efe739f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.439028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da32712-4ac8-4f0e-925b-cdbcdec95157 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.449498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da32712-4ac8-4f0e-925b-cdbcdec95157 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.454719 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1ca68-ccb1-4769-a70f-46ff9a810086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.455403 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1ca68-ccb1-4769-a70f-46ff9a810086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.455683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1ca68-ccb1-4769-a70f-46ff9a810086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.455963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1ca68-ccb1-4769-a70f-46ff9a810086 admin admin] PATH_INFO: `/v3/projects/d1fb9b75e13f44338055e3d1731bff87/users/2b093450f7c54d8ea0b6e1471990b486/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.457415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c8f85634-b4f3-408c-9e2a-699efe739f4a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:57.458629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c8f85634-b4f3-408c-9e2a-699efe739f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:57.467831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1242/2485] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:57.471899 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1ca68-ccb1-4769-a70f-46ff9a810086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1fb9b75e13f44338055e3d1731bff87, user_id=2b093450f7c54d8ea0b6e1471990b486, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.472060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f85634-b4f3-408c-9e2a-699efe739f4a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.474591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1ca68-ccb1-4769-a70f-46ff9a810086 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.480862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f85634-b4f3-408c-9e2a-699efe739f4a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.486497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.487167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.487443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.487709 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.488266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1244/2486] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:57 2022] PUT /identity/v3/projects/d1fb9b75e13f44338055e3d1731bff87/users/2b093450f7c54d8ea0b6e1471990b486/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:57.488372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:05:57.488695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.490417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.492155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1ca68-ccb1-4769-a70f-46ff9a810086 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.505895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1243/2487] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:05:57.506988 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.518103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c251b2a-7b5b-4773-a420-4bfe16a4b63e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.557265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1ca68-ccb1-4769-a70f-46ff9a810086 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.561887 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203541d-60e6-4dee-a1a1-0b15ca66a119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.562478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203541d-60e6-4dee-a1a1-0b15ca66a119 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.562756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203541d-60e6-4dee-a1a1-0b15ca66a119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.563101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203541d-60e6-4dee-a1a1-0b15ca66a119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.563694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203541d-60e6-4dee-a1a1-0b15ca66a119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.565280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203541d-60e6-4dee-a1a1-0b15ca66a119 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.575198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabd2b-4431-4673-8392-2f31551fa1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.575512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1245/2488] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:05:57.575807 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabd2b-4431-4673-8392-2f31551fa1bd admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.576119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabd2b-4431-4673-8392-2f31551fa1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.576395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabd2b-4431-4673-8392-2f31551fa1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.577022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fabd2b-4431-4673-8392-2f31551fa1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.578989 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fabd2b-4431-4673-8392-2f31551fa1bd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.579278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203541d-60e6-4dee-a1a1-0b15ca66a119 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.589043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203541d-60e6-4dee-a1a1-0b15ca66a119 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.594117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866d1f-569a-456c-981a-c8afcc092417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.594830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866d1f-569a-456c-981a-c8afcc092417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.595148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866d1f-569a-456c-981a-c8afcc092417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.595422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866d1f-569a-456c-981a-c8afcc092417 admin admin] PATH_INFO: `/v3/projects/d1fb9b75e13f44338055e3d1731bff87/users/2b093450f7c54d8ea0b6e1471990b486/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.597538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26866d1f-569a-456c-981a-c8afcc092417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1fb9b75e13f44338055e3d1731bff87, user_id=2b093450f7c54d8ea0b6e1471990b486, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.600740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26866d1f-569a-456c-981a-c8afcc092417 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.607913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1244/2489] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:57.611958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabd2b-4431-4673-8392-2f31551fa1bd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.612216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1246/2490] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:57 2022] PUT /identity/v3/projects/d1fb9b75e13f44338055e3d1731bff87/users/2b093450f7c54d8ea0b6e1471990b486/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:57.623346 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:57.623676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c762342-f3a0-439b-be16-57341eca368c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.623963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c762342-f3a0-439b-be16-57341eca368c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.624233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c762342-f3a0-439b-be16-57341eca368c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.653713 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-1c762342-f3a0-439b-be16-57341eca368c None None] MFA Rules not processed for user `2b093450f7c54d8ea0b6e1471990b486`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:57.654973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1c762342-f3a0-439b-be16-57341eca368c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:57.682359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabd2b-4431-4673-8392-2f31551fa1bd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.687189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039aa52-89ed-4185-adfe-cc6728e686b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.687781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039aa52-89ed-4185-adfe-cc6728e686b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.688082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039aa52-89ed-4185-adfe-cc6728e686b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.688348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039aa52-89ed-4185-adfe-cc6728e686b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.688967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039aa52-89ed-4185-adfe-cc6728e686b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.690880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039aa52-89ed-4185-adfe-cc6728e686b2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.714295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1245/2491] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:05:57.715577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039aa52-89ed-4185-adfe-cc6728e686b2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.723737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1247/2492] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:57.725004 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039aa52-89ed-4185-adfe-cc6728e686b2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.730517 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fb9e5-a3f4-4903-ba74-9ca957d120d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.731126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fb9e5-a3f4-4903-ba74-9ca957d120d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.731403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fb9e5-a3f4-4903-ba74-9ca957d120d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.731672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fb9e5-a3f4-4903-ba74-9ca957d120d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.732301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fb9e5-a3f4-4903-ba74-9ca957d120d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.734162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fb9e5-a3f4-4903-ba74-9ca957d120d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.734545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c762342-f3a0-439b-be16-57341eca368c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.735897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1c762342-f3a0-439b-be16-57341eca368c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:57.747638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1246/2493] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:05:57.748583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fb9e5-a3f4-4903-ba74-9ca957d120d3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.758059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fb9e5-a3f4-4903-ba74-9ca957d120d3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.762688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f982f-beed-4c34-a181-e2b6c68ca1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.763286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f982f-beed-4c34-a181-e2b6c68ca1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.763566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f982f-beed-4c34-a181-e2b6c68ca1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.763834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f982f-beed-4c34-a181-e2b6c68ca1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.764453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f982f-beed-4c34-a181-e2b6c68ca1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.765994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f982f-beed-4c34-a181-e2b6c68ca1a8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.770216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:57.777433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1247/2494] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:57.781914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f982f-beed-4c34-a181-e2b6c68ca1a8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:57.792302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f982f-beed-4c34-a181-e2b6c68ca1a8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.794233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c762342-f3a0-439b-be16-57341eca368c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:57.796938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87fabe-a205-4f59-b3d6-c389a0009072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.797568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87fabe-a205-4f59-b3d6-c389a0009072 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.797851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87fabe-a205-4f59-b3d6-c389a0009072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.797955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7b899-8bdf-4d44-8428-2215d63635de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:57.798120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87fabe-a205-4f59-b3d6-c389a0009072 admin admin] PATH_INFO: `/v3/projects/61d8c6e9197648b0baa61978ca6da29f/users/241d9dc10ecc499891dfa98be822e2c8/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.798728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7b899-8bdf-4d44-8428-2215d63635de service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.799051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7b899-8bdf-4d44-8428-2215d63635de service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.799327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7b899-8bdf-4d44-8428-2215d63635de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.809080 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7b899-8bdf-4d44-8428-2215d63635de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.810511 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7b899-8bdf-4d44-8428-2215d63635de service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.813725 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87fabe-a205-4f59-b3d6-c389a0009072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d8c6e9197648b0baa61978ca6da29f, user_id=241d9dc10ecc499891dfa98be822e2c8, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:57.817066 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87fabe-a205-4f59-b3d6-c389a0009072 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:57.820316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1248/2495] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:05:57.832121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1248/2496] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:57 2022] PUT /identity/v3/projects/61d8c6e9197648b0baa61978ca6da29f/users/241d9dc10ecc499891dfa98be822e2c8/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:05:57.842880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130e701-f582-4e89-ba53-88ebc3d9ad68 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:57.843224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130e701-f582-4e89-ba53-88ebc3d9ad68 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:57.843521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130e701-f582-4e89-ba53-88ebc3d9ad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:57.874259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9130e701-f582-4e89-ba53-88ebc3d9ad68 None None] MFA Rules not processed for user `241d9dc10ecc499891dfa98be822e2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:57.875525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9130e701-f582-4e89-ba53-88ebc3d9ad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:57.947322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1249/2497] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:57.951721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87fabe-a205-4f59-b3d6-c389a0009072 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:58.029864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87fabe-a205-4f59-b3d6-c389a0009072 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:58.035111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b5f0-f0be-4fa2-9959-809e5d875a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:58.035829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b5f0-f0be-4fa2-9959-809e5d875a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.036214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b5f0-f0be-4fa2-9959-809e5d875a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.036580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b5f0-f0be-4fa2-9959-809e5d875a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.037497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55b5f0-f0be-4fa2-9959-809e5d875a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:58.040290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55b5f0-f0be-4fa2-9959-809e5d875a64 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:58.084196 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1249/2498] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:05:57 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:58.088846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130e701-f582-4e89-ba53-88ebc3d9ad68 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:58.109808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130e701-f582-4e89-ba53-88ebc3d9ad68 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:58.208348 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643caaa-4ed3-41f7-8775-81d65f5a2b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:58.208971 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643caaa-4ed3-41f7-8775-81d65f5a2b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.209255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643caaa-4ed3-41f7-8775-81d65f5a2b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.209523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643caaa-4ed3-41f7-8775-81d65f5a2b2d admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.210202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643caaa-4ed3-41f7-8775-81d65f5a2b2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:58.212728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643caaa-4ed3-41f7-8775-81d65f5a2b2d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:58.242923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1250/2499] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:05:58 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:05:58.245723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b5f0-f0be-4fa2-9959-809e5d875a64 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:58.256218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b5f0-f0be-4fa2-9959-809e5d875a64 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:58.263500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86e297-93cd-4326-9ac6-78546aa279c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:58.264209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86e297-93cd-4326-9ac6-78546aa279c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.264555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86e297-93cd-4326-9ac6-78546aa279c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.264830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86e297-93cd-4326-9ac6-78546aa279c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.265504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86e297-93cd-4326-9ac6-78546aa279c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:58.267100 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86e297-93cd-4326-9ac6-78546aa279c4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:58.279241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1250/2500] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:05:58.283515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643caaa-4ed3-41f7-8775-81d65f5a2b2d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:58.293938 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643caaa-4ed3-41f7-8775-81d65f5a2b2d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:58.301044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5593-6e52-4c10-b1cd-ba3ae6e02e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:58.301919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5593-6e52-4c10-b1cd-ba3ae6e02e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.302410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5593-6e52-4c10-b1cd-ba3ae6e02e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.302923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5593-6e52-4c10-b1cd-ba3ae6e02e22 admin admin] PATH_INFO: `/v3/projects/6218a4cacbe345d09800912e85d69fa2/users/82bf4c6c26b74a0098b8a0b94a6bcc5f/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.322499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c5593-6e52-4c10-b1cd-ba3ae6e02e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6218a4cacbe345d09800912e85d69fa2, user_id=82bf4c6c26b74a0098b8a0b94a6bcc5f, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:58.326071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c5593-6e52-4c10-b1cd-ba3ae6e02e22 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:58.345368 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1251/2501] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:58 2022] PUT /identity/v3/projects/6218a4cacbe345d09800912e85d69fa2/users/82bf4c6c26b74a0098b8a0b94a6bcc5f/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:58.349169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86e297-93cd-4326-9ac6-78546aa279c4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:58.360982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86e297-93cd-4326-9ac6-78546aa279c4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:58.391015 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:58.453213 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23315a1b-e356-4665-8f1b-3d241b8ad182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:58.453840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315a1b-e356-4665-8f1b-3d241b8ad182 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.454111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315a1b-e356-4665-8f1b-3d241b8ad182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.454384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315a1b-e356-4665-8f1b-3d241b8ad182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.458275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23315a1b-e356-4665-8f1b-3d241b8ad182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:58.459965 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23315a1b-e356-4665-8f1b-3d241b8ad182 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:58.474945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1251/2502] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:05:58.476610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5593-6e52-4c10-b1cd-ba3ae6e02e22 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:58.489287 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5593-6e52-4c10-b1cd-ba3ae6e02e22 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:58.494863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df059b6d-09fd-4dd3-a33a-a3cf06da6c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:58.495524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059b6d-09fd-4dd3-a33a-a3cf06da6c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.495807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059b6d-09fd-4dd3-a33a-a3cf06da6c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.496128 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059b6d-09fd-4dd3-a33a-a3cf06da6c97 admin admin] PATH_INFO: `/v3/projects/6218a4cacbe345d09800912e85d69fa2/users/82bf4c6c26b74a0098b8a0b94a6bcc5f/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.498644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df059b6d-09fd-4dd3-a33a-a3cf06da6c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6218a4cacbe345d09800912e85d69fa2, user_id=82bf4c6c26b74a0098b8a0b94a6bcc5f, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:58.501384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df059b6d-09fd-4dd3-a33a-a3cf06da6c97 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:58.513622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1252/2503] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:05:58 2022] PUT /identity/v3/projects/6218a4cacbe345d09800912e85d69fa2/users/82bf4c6c26b74a0098b8a0b94a6bcc5f/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:58.523651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d316a1-df3c-4117-a5c9-a9e551ccbbfc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.524010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d316a1-df3c-4117-a5c9-a9e551ccbbfc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.524295 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d316a1-df3c-4117-a5c9-a9e551ccbbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.555431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-93d316a1-df3c-4117-a5c9-a9e551ccbbfc None None] MFA Rules not processed for user `82bf4c6c26b74a0098b8a0b94a6bcc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:58.556657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-93d316a1-df3c-4117-a5c9-a9e551ccbbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:58.644025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1252/2504] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:58.656072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df059b6d-09fd-4dd3-a33a-a3cf06da6c97 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:58.686224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:58.731630 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df059b6d-09fd-4dd3-a33a-a3cf06da6c97 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:58.735136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3b867-b07d-4948-8a02-56f833f3cf44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:58.735734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3b867-b07d-4948-8a02-56f833f3cf44 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.736009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3b867-b07d-4948-8a02-56f833f3cf44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.736295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3b867-b07d-4948-8a02-56f833f3cf44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.746015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3b867-b07d-4948-8a02-56f833f3cf44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:58.747697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3b867-b07d-4948-8a02-56f833f3cf44 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:58.757789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1253/2505] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:59.171862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d316a1-df3c-4117-a5c9-a9e551ccbbfc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:59.285548 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d316a1-df3c-4117-a5c9-a9e551ccbbfc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:59.291068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:59.292153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:59.292630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:59.293063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] PATH_INFO: `/v3/users/241d9dc10ecc499891dfa98be822e2c8` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:59.294513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] RBAC: Authorizing `identity:delete_user(user_id=241d9dc10ecc499891dfa98be822e2c8)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:59.297067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:59.357727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '241d9dc10ecc499891dfa98be822e2c8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.359625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:59.368669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '241d9dc10ecc499891dfa98be822e2c8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.369745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '241d9dc10ecc499891dfa98be822e2c8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.370542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '241d9dc10ecc499891dfa98be822e2c8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.379542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7de0c305-f686-4cde-aea6-b5efcce9917e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '241d9dc10ecc499891dfa98be822e2c8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.405955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3b867-b07d-4948-8a02-56f833f3cf44 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:59.408320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1253/2506] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:59 2022] DELETE /identity/v3/users/241d9dc10ecc499891dfa98be822e2c8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:59.412531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5dc4-607d-452c-9df2-3e3a25c73525 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:59.412953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5dc4-607d-452c-9df2-3e3a25c73525 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:59.413232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5dc4-607d-452c-9df2-3e3a25c73525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:59.420671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3b867-b07d-4948-8a02-56f833f3cf44 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:59.434786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-535e5dc4-607d-452c-9df2-3e3a25c73525 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:59.436032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-535e5dc4-607d-452c-9df2-3e3a25c73525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:59.486794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:59.487562 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:59.487866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:59.488142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] PATH_INFO: `/v3/users/0a29181eff26432195bde123afe9f556` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:59.489233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a29181eff26432195bde123afe9f556)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:59.496190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:59.498802 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1254/2507] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:59.509641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5dc4-607d-452c-9df2-3e3a25c73525 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:59.511910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-535e5dc4-607d-452c-9df2-3e3a25c73525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:59.548832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a29181eff26432195bde123afe9f556'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.556459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a29181eff26432195bde123afe9f556'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.557252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a29181eff26432195bde123afe9f556'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.557948 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a29181eff26432195bde123afe9f556'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.567324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-809591b5-9ec2-412f-8408-8749e8fdfb59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a29181eff26432195bde123afe9f556'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:05:59.581852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5dc4-607d-452c-9df2-3e3a25c73525 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:59.586577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:59.587350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:59.587625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:59.587914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:59.588545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1254/2508] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:05:59 2022] DELETE /identity/v3/users/0a29181eff26432195bde123afe9f556 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:05:59.593392 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651eb9e-edfa-4be7-9509-bbee20b60366 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:59.593635 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:59.593784 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651eb9e-edfa-4be7-9509-bbee20b60366 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:59.593950 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651eb9e-edfa-4be7-9509-bbee20b60366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:59.606440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:59.607958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:59.615212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a651eb9e-edfa-4be7-9509-bbee20b60366 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:59.616405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a651eb9e-edfa-4be7-9509-bbee20b60366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:59.618707 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1255/2509] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:05:59.654184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:59.688701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1255/2510] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:59.700615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:59.702860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:59.721669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ae5a-bf15-4a0f-9349-859cac2bb6ab service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:59.781686 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a811ef-6242-4b00-8c16-507c7c215549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:59.782304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a811ef-6242-4b00-8c16-507c7c215549 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:59.782596 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a811ef-6242-4b00-8c16-507c7c215549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:59.783119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a811ef-6242-4b00-8c16-507c7c215549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:59.798006 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a811ef-6242-4b00-8c16-507c7c215549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:59.799775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a811ef-6242-4b00-8c16-507c7c215549 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:59.821873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1256/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:59.927345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651eb9e-edfa-4be7-9509-bbee20b60366 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:59.929032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a651eb9e-edfa-4be7-9509-bbee20b60366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:59.949423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651eb9e-edfa-4be7-9509-bbee20b60366 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:59.962604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:59.963307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:59.963999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:59.963999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] PATH_INFO: `/v3/projects/61d8c6e9197648b0baa61978ca6da29f` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:59.965361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d8c6e9197648b0baa61978ca6da29f)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:59.967472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:00.040727 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d8c6e9197648b0baa61978ca6da29f', 'request_id': 'req-e1ef21e1-830c-4b50-a5f1-6d238f24f024'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.049800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d8c6e9197648b0baa61978ca6da29f', 'request_id': 'req-e1ef21e1-830c-4b50-a5f1-6d238f24f024'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.056495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d8c6e9197648b0baa61978ca6da29f', 'request_id': 'req-e1ef21e1-830c-4b50-a5f1-6d238f24f024'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.068131 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] The token cache is being invalidate because project 61d8c6e9197648b0baa61978ca6da29f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:00.068421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.069056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.070539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1256/2512] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:05:59 2022] DELETE /identity/v3/projects/61d8c6e9197648b0baa61978ca6da29f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:00.074433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a811ef-6242-4b00-8c16-507c7c215549 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:00.084662 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a811ef-6242-4b00-8c16-507c7c215549 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:00.096234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:00.145502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:00.147168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:00.170656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:00.172075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:00.172731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:00.172731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] PATH_INFO: `/v3/users/82bf4c6c26b74a0098b8a0b94a6bcc5f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:00.174497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bf4c6c26b74a0098b8a0b94a6bcc5f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:00.176485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:00.236668 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bf4c6c26b74a0098b8a0b94a6bcc5f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.236897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef21e1-830c-4b50-a5f1-6d238f24f024 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:00.246818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bf4c6c26b74a0098b8a0b94a6bcc5f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.247563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bf4c6c26b74a0098b8a0b94a6bcc5f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.247761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:00.248371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:00.248646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:00.248914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] PATH_INFO: `/v3/projects/bd1c0d6ad536480fb7d50ae3dd7571b2` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:00.249313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bf4c6c26b74a0098b8a0b94a6bcc5f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.250112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1c0d6ad536480fb7d50ae3dd7571b2)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:00.252036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:00.263492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bf4c6c26b74a0098b8a0b94a6bcc5f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.293021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1257/2513] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:00 2022] DELETE /identity/v3/users/82bf4c6c26b74a0098b8a0b94a6bcc5f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:00.310524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1c0d6ad536480fb7d50ae3dd7571b2', 'request_id': 'req-f1ed63e1-88c9-4a27-99fd-6f1524201732'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.321289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1c0d6ad536480fb7d50ae3dd7571b2', 'request_id': 'req-f1ed63e1-88c9-4a27-99fd-6f1524201732'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.329941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1c0d6ad536480fb7d50ae3dd7571b2', 'request_id': 'req-f1ed63e1-88c9-4a27-99fd-6f1524201732'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.334827 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:00.344195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] The token cache is being invalidate because project bd1c0d6ad536480fb7d50ae3dd7571b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:00.346614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.347787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.353224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1257/2514] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:00 2022] DELETE /identity/v3/projects/bd1c0d6ad536480fb7d50ae3dd7571b2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:00.354567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:00.356485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:00.440097 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95590dc-589e-470c-a31f-37d39a7c3a46 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:00.481449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:00.513293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:00.514117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:00.514400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:00.514679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] PATH_INFO: `/v3/users/2b093450f7c54d8ea0b6e1471990b486` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:00.516288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b093450f7c54d8ea0b6e1471990b486)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:00.519456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:00.579895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b093450f7c54d8ea0b6e1471990b486'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.594596 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b093450f7c54d8ea0b6e1471990b486'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.595367 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b093450f7c54d8ea0b6e1471990b486'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.595996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b093450f7c54d8ea0b6e1471990b486'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.605162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b093450f7c54d8ea0b6e1471990b486'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.627684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1258/2515] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:00 2022] DELETE /identity/v3/users/2b093450f7c54d8ea0b6e1471990b486 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:00.768108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:00.769770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:00.786007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed63e1-88c9-4a27-99fd-6f1524201732 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:00.860084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:00.861255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:00.861482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:00.861753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] PATH_INFO: `/v3/projects/6218a4cacbe345d09800912e85d69fa2` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:00.862884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6218a4cacbe345d09800912e85d69fa2)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:00.865319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:00.926321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6218a4cacbe345d09800912e85d69fa2', 'request_id': 'req-c202d4e6-8e30-4c88-8ef1-5ccd42712531'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.933237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:00.934870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:00.936708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6218a4cacbe345d09800912e85d69fa2', 'request_id': 'req-c202d4e6-8e30-4c88-8ef1-5ccd42712531'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.946089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6218a4cacbe345d09800912e85d69fa2', 'request_id': 'req-c202d4e6-8e30-4c88-8ef1-5ccd42712531'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.955637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bb998-85ba-4f05-829e-bb1ed19f8c6f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:00.959121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] The token cache is being invalidate because project 6218a4cacbe345d09800912e85d69fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:00.959435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.960175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-c202d4e6-8e30-4c88-8ef1-5ccd42712531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:00.962476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1258/2516] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:00 2022] DELETE /identity/v3/projects/6218a4cacbe345d09800912e85d69fa2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:00.967390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e098f-d0ad-43be-9615-ce54420edcc9 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:00.967685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e098f-d0ad-43be-9615-ce54420edcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:00.967957 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e098f-d0ad-43be-9615-ce54420edcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:00.991852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-685e098f-d0ad-43be-9615-ce54420edcc9 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:00.993990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-685e098f-d0ad-43be-9615-ce54420edcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:01.035929 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.036704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.037110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.037484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] PATH_INFO: `/v3/projects/d1fb9b75e13f44338055e3d1731bff87` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.038730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1fb9b75e13f44338055e3d1731bff87)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.041311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.071991 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1259/2517] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:01.082050 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff0c1-13f0-432f-af7c-1331abc9701d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.082426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff0c1-13f0-432f-af7c-1331abc9701d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.082727 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff0c1-13f0-432f-af7c-1331abc9701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.094519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1fb9b75e13f44338055e3d1731bff87', 'request_id': 'req-062a0eca-6eeb-4e21-a38b-56971b2676b6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:01.103958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-4adff0c1-13f0-432f-af7c-1331abc9701d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:01.105168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4adff0c1-13f0-432f-af7c-1331abc9701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:01.106423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1fb9b75e13f44338055e3d1731bff87', 'request_id': 'req-062a0eca-6eeb-4e21-a38b-56971b2676b6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:01.109492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:01.116492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1fb9b75e13f44338055e3d1731bff87', 'request_id': 'req-062a0eca-6eeb-4e21-a38b-56971b2676b6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:01.129069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:01.130013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] The token cache is being invalidate because project d1fb9b75e13f44338055e3d1731bff87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:01.130422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:01.131200 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-062a0eca-6eeb-4e21-a38b-56971b2676b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:01.136331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1259/2518] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:00 2022] DELETE /identity/v3/projects/d1fb9b75e13f44338055e3d1731bff87 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:01.138282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef194a-141d-4974-b0b9-5e3d6cad9ed0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.138522 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef194a-141d-4974-b0b9-5e3d6cad9ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.138801 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef194a-141d-4974-b0b9-5e3d6cad9ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.160828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-eaef194a-141d-4974-b0b9-5e3d6cad9ed0 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:01.162065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eaef194a-141d-4974-b0b9-5e3d6cad9ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:01.180865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1260/2519] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:01.185000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adff0c1-13f0-432f-af7c-1331abc9701d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.194956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adff0c1-13f0-432f-af7c-1331abc9701d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.201706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.201926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.202203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.202471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.203169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:01.203466 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.205052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.217686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1261/2520] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:06:01.222029 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.231307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1260/2521] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:01.232831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cb856-b3e9-441a-bd9c-601ad2e0c712 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.237349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4a3be-1907-4697-a55c-696657c8ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.237937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4a3be-1907-4697-a55c-696657c8ce7d admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.238211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4a3be-1907-4697-a55c-696657c8ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.238475 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4a3be-1907-4697-a55c-696657c8ce7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.239149 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4a3be-1907-4697-a55c-696657c8ce7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.239278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aec2d-b6b1-4f63-b9be-70d85fcc2e73 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.239511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aec2d-b6b1-4f63-b9be-70d85fcc2e73 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.239784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aec2d-b6b1-4f63-b9be-70d85fcc2e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.241038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4a3be-1907-4697-a55c-696657c8ce7d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.259991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-883aec2d-b6b1-4f63-b9be-70d85fcc2e73 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:01.261386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-883aec2d-b6b1-4f63-b9be-70d85fcc2e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:01.271672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1262/2522] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:01.275813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4a3be-1907-4697-a55c-696657c8ce7d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.276232 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1261/2523] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:01.280404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aec2d-b6b1-4f63-b9be-70d85fcc2e73 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.290323 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aec2d-b6b1-4f63-b9be-70d85fcc2e73 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.354866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4a3be-1907-4697-a55c-696657c8ce7d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.361187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e732b5-9b7c-47c9-86cc-23ab8bdc99e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.361912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e732b5-9b7c-47c9-86cc-23ab8bdc99e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.362298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e732b5-9b7c-47c9-86cc-23ab8bdc99e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.362670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e732b5-9b7c-47c9-86cc-23ab8bdc99e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.363430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e732b5-9b7c-47c9-86cc-23ab8bdc99e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.365708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e732b5-9b7c-47c9-86cc-23ab8bdc99e4 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.386103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.386732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.387195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.387470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.388122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:01.388423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.390053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.393536 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1263/2524] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/users => generated 282 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:01.394638 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e732b5-9b7c-47c9-86cc-23ab8bdc99e4 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.406352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1262/2525] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:06:01.408133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.421343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c5569-9c8a-423c-bc56-0d545cb5a52f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.424528 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e732b5-9b7c-47c9-86cc-23ab8bdc99e4 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.426856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a5ed3-14cb-4895-92e0-ba69897d4bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.427615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a5ed3-14cb-4895-92e0-ba69897d4bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.427901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a5ed3-14cb-4895-92e0-ba69897d4bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.428208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a5ed3-14cb-4895-92e0-ba69897d4bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.428866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a5ed3-14cb-4895-92e0-ba69897d4bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.430278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e79cf-1546-4747-91d2-4ab78e80ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.431076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e79cf-1546-4747-91d2-4ab78e80ef67 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.431381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e79cf-1546-4747-91d2-4ab78e80ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.431491 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a5ed3-14cb-4895-92e0-ba69897d4bf8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.431783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e79cf-1546-4747-91d2-4ab78e80ef67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.432522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e79cf-1546-4747-91d2-4ab78e80ef67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.434129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e79cf-1546-4747-91d2-4ab78e80ef67 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.445446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1264/2526] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:01.449438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e79cf-1546-4747-91d2-4ab78e80ef67 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.460443 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e79cf-1546-4747-91d2-4ab78e80ef67 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.463184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1263/2527] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:01.467671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a5ed3-14cb-4895-92e0-ba69897d4bf8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.526143 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52736158-9bb7-46e2-81eb-7351109bf164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.526748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736158-9bb7-46e2-81eb-7351109bf164 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.527064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736158-9bb7-46e2-81eb-7351109bf164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.527332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52736158-9bb7-46e2-81eb-7351109bf164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.527973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52736158-9bb7-46e2-81eb-7351109bf164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.529583 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52736158-9bb7-46e2-81eb-7351109bf164 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.532121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:01.544216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1265/2528] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:01.545385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52736158-9bb7-46e2-81eb-7351109bf164 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.550687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a5ed3-14cb-4895-92e0-ba69897d4bf8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.552997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:01.556158 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700abdb2-c058-4044-be57-c2a8fb66e0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.556787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700abdb2-c058-4044-be57-c2a8fb66e0fc admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.557093 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700abdb2-c058-4044-be57-c2a8fb66e0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.557341 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700abdb2-c058-4044-be57-c2a8fb66e0fc admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.557597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52736158-9bb7-46e2-81eb-7351109bf164 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.558262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700abdb2-c058-4044-be57-c2a8fb66e0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.560250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700abdb2-c058-4044-be57-c2a8fb66e0fc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.563962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b1b78-1d42-4ef7-a3fc-19cee8c2fda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.564806 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b1b78-1d42-4ef7-a3fc-19cee8c2fda5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.564957 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b1b78-1d42-4ef7-a3fc-19cee8c2fda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.565227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b1b78-1d42-4ef7-a3fc-19cee8c2fda5 admin admin] PATH_INFO: `/v3/projects/97be096875584107a464610810b6be53/users/b7ba9104ffd94389b72ad9aed27dc3eb/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.581457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b1b78-1d42-4ef7-a3fc-19cee8c2fda5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97be096875584107a464610810b6be53, user_id=b7ba9104ffd94389b72ad9aed27dc3eb, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.584125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b1b78-1d42-4ef7-a3fc-19cee8c2fda5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.585044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1264/2529] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:01.589871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700abdb2-c058-4044-be57-c2a8fb66e0fc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.598107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1266/2530] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:01 2022] PUT /identity/v3/projects/97be096875584107a464610810b6be53/users/b7ba9104ffd94389b72ad9aed27dc3eb/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:01.601017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700abdb2-c058-4044-be57-c2a8fb66e0fc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.608876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6bc48-8860-4682-ac4d-139085403e1a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.609442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6bc48-8860-4682-ac4d-139085403e1a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.610094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6bc48-8860-4682-ac4d-139085403e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.637548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-76b6bc48-8860-4682-ac4d-139085403e1a None None] MFA Rules not processed for user `b7ba9104ffd94389b72ad9aed27dc3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:01.638753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-76b6bc48-8860-4682-ac4d-139085403e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:01.658172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e9214-35a6-43e0-a266-a7c58145ec3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.658785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e9214-35a6-43e0-a266-a7c58145ec3e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.659098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e9214-35a6-43e0-a266-a7c58145ec3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.659374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e9214-35a6-43e0-a266-a7c58145ec3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.660031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e9214-35a6-43e0-a266-a7c58145ec3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.661677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e9214-35a6-43e0-a266-a7c58145ec3e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.672528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1265/2531] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:01.676735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e9214-35a6-43e0-a266-a7c58145ec3e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.686821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e9214-35a6-43e0-a266-a7c58145ec3e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.691425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c18faa-cd7c-4f12-ad4c-caef750a9c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.692008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c18faa-cd7c-4f12-ad4c-caef750a9c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.692290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c18faa-cd7c-4f12-ad4c-caef750a9c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.692566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c18faa-cd7c-4f12-ad4c-caef750a9c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.693197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c18faa-cd7c-4f12-ad4c-caef750a9c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.694861 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c18faa-cd7c-4f12-ad4c-caef750a9c88 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.704645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1267/2532] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:01.706209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1266/2533] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:01.709235 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6bc48-8860-4682-ac4d-139085403e1a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.712489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c18faa-cd7c-4f12-ad4c-caef750a9c88 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.722139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6bc48-8860-4682-ac4d-139085403e1a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.723362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c18faa-cd7c-4f12-ad4c-caef750a9c88 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.727968 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d95ba-665f-4425-a81f-6b7e921f6769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.728686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d95ba-665f-4425-a81f-6b7e921f6769 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.728806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a463429-5518-421a-a2f6-1564aaa59be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.729102 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d95ba-665f-4425-a81f-6b7e921f6769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.729445 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d95ba-665f-4425-a81f-6b7e921f6769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.729630 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a463429-5518-421a-a2f6-1564aaa59be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.729974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a463429-5518-421a-a2f6-1564aaa59be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.730276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d95ba-665f-4425-a81f-6b7e921f6769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.730395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a463429-5518-421a-a2f6-1564aaa59be0 admin admin] PATH_INFO: `/v3/projects/dd6bce1a0b164725b24466255767c8c5/users/74efc33a0d444dcc9075dcfdb1f4cdce/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.733711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d95ba-665f-4425-a81f-6b7e921f6769 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.747290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a463429-5518-421a-a2f6-1564aaa59be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6bce1a0b164725b24466255767c8c5, user_id=74efc33a0d444dcc9075dcfdb1f4cdce, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.749997 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a463429-5518-421a-a2f6-1564aaa59be0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.763720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1268/2534] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:01.767604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d95ba-665f-4425-a81f-6b7e921f6769 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.768090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1267/2535] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:01 2022] PUT /identity/v3/projects/dd6bce1a0b164725b24466255767c8c5/users/74efc33a0d444dcc9075dcfdb1f4cdce/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:01.779206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7cf5-8994-4245-839a-1cc23e0477a6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.779494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7cf5-8994-4245-839a-1cc23e0477a6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.779772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7cf5-8994-4245-839a-1cc23e0477a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.809845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ea6a7cf5-8994-4245-839a-1cc23e0477a6 None None] MFA Rules not processed for user `74efc33a0d444dcc9075dcfdb1f4cdce`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:01.811327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ea6a7cf5-8994-4245-839a-1cc23e0477a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:01.846119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d95ba-665f-4425-a81f-6b7e921f6769 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.853897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd8002-6f8d-4131-966f-ee22ed1c10d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.855117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd8002-6f8d-4131-966f-ee22ed1c10d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.855517 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd8002-6f8d-4131-966f-ee22ed1c10d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.855889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd8002-6f8d-4131-966f-ee22ed1c10d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.857057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd8002-6f8d-4131-966f-ee22ed1c10d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.859450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd8002-6f8d-4131-966f-ee22ed1c10d9 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.881366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1268/2536] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:01.888302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1269/2537] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:01.889020 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a7cf5-8994-4245-839a-1cc23e0477a6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.889269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd8002-6f8d-4131-966f-ee22ed1c10d9 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.900598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd8002-6f8d-4131-966f-ee22ed1c10d9 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.904172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a7cf5-8994-4245-839a-1cc23e0477a6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.905887 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472ebdd-e980-4659-8bdb-dbc958a24372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.906590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472ebdd-e980-4659-8bdb-dbc958a24372 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.906997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472ebdd-e980-4659-8bdb-dbc958a24372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.907386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472ebdd-e980-4659-8bdb-dbc958a24372 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.908159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472ebdd-e980-4659-8bdb-dbc958a24372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.910070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412bf4c-4f7c-4baa-b66c-3faea0e882fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.910203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472ebdd-e980-4659-8bdb-dbc958a24372 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.910766 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412bf4c-4f7c-4baa-b66c-3faea0e882fb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.911607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412bf4c-4f7c-4baa-b66c-3faea0e882fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.911607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412bf4c-4f7c-4baa-b66c-3faea0e882fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.912852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412bf4c-4f7c-4baa-b66c-3faea0e882fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.913992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412bf4c-4f7c-4baa-b66c-3faea0e882fb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.925914 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1269/2538] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:01.936035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412bf4c-4f7c-4baa-b66c-3faea0e882fb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.948526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412bf4c-4f7c-4baa-b66c-3faea0e882fb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.953974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2f5bd-bc4f-426d-b899-6a71dc8cc898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.954181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1270/2539] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:01.954647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f5bd-bc4f-426d-b899-6a71dc8cc898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.955470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f5bd-bc4f-426d-b899-6a71dc8cc898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.955470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f5bd-bc4f-426d-b899-6a71dc8cc898 admin admin] PATH_INFO: `/v3/projects/4a44d82f6a284d95ae40914309cda99c/users/0af6f42a8786493f94565159cdb30213/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.958359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472ebdd-e980-4659-8bdb-dbc958a24372 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.973388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2f5bd-bc4f-426d-b899-6a71dc8cc898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a44d82f6a284d95ae40914309cda99c, user_id=0af6f42a8786493f94565159cdb30213, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.976250 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2f5bd-bc4f-426d-b899-6a71dc8cc898 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.981062 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:01.988793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:01.995785 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1270/2540] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:01 2022] PUT /identity/v3/projects/4a44d82f6a284d95ae40914309cda99c/users/0af6f42a8786493f94565159cdb30213/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:01.999842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2f5bd-bc4f-426d-b899-6a71dc8cc898 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.032089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472ebdd-e980-4659-8bdb-dbc958a24372 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.070595 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2f5bd-bc4f-426d-b899-6a71dc8cc898 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.077443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2af2dd-560b-48e4-b428-0bf19c5ce1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.078263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2af2dd-560b-48e4-b428-0bf19c5ce1f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.078607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2af2dd-560b-48e4-b428-0bf19c5ce1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.078967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2af2dd-560b-48e4-b428-0bf19c5ce1f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.080234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2af2dd-560b-48e4-b428-0bf19c5ce1f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.082131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2af2dd-560b-48e4-b428-0bf19c5ce1f9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.093647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1271/2541] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:02.096187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9b00f-7f81-48c4-bdf2-d11670c5b623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.097080 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9b00f-7f81-48c4-bdf2-d11670c5b623 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.097402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9b00f-7f81-48c4-bdf2-d11670c5b623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.097572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2af2dd-560b-48e4-b428-0bf19c5ce1f9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.097693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9b00f-7f81-48c4-bdf2-d11670c5b623 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.098233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9b00f-7f81-48c4-bdf2-d11670c5b623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.100147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9b00f-7f81-48c4-bdf2-d11670c5b623 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.108107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2af2dd-560b-48e4-b428-0bf19c5ce1f9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.116182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081aade-8d51-4b7c-a784-9b3a8dd4249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.116867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081aade-8d51-4b7c-a784-9b3a8dd4249a admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.117681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081aade-8d51-4b7c-a784-9b3a8dd4249a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.117953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081aade-8d51-4b7c-a784-9b3a8dd4249a admin admin] PATH_INFO: `/v3/projects/4a44d82f6a284d95ae40914309cda99c/users/0af6f42a8786493f94565159cdb30213/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.120012 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081aade-8d51-4b7c-a784-9b3a8dd4249a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a44d82f6a284d95ae40914309cda99c, user_id=0af6f42a8786493f94565159cdb30213, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.122772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081aade-8d51-4b7c-a784-9b3a8dd4249a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.126469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1271/2542] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:01 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:02.129749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9b00f-7f81-48c4-bdf2-d11670c5b623 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.135336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1272/2543] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:02 2022] PUT /identity/v3/projects/4a44d82f6a284d95ae40914309cda99c/users/0af6f42a8786493f94565159cdb30213/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:02.145158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9b00f-7f81-48c4-bdf2-d11670c5b623 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.147079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7933041-450b-435d-9885-90d52de41f0f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.147079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7933041-450b-435d-9885-90d52de41f0f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.147079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7933041-450b-435d-9885-90d52de41f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.180800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f7933041-450b-435d-9885-90d52de41f0f None None] MFA Rules not processed for user `0af6f42a8786493f94565159cdb30213`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:02.182475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f7933041-450b-435d-9885-90d52de41f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:02.209023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927dc8fa-f281-45fb-adac-dae427b7c820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.209603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dc8fa-f281-45fb-adac-dae427b7c820 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.209902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dc8fa-f281-45fb-adac-dae427b7c820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.210457 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dc8fa-f281-45fb-adac-dae427b7c820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.210834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927dc8fa-f281-45fb-adac-dae427b7c820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.212502 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927dc8fa-f281-45fb-adac-dae427b7c820 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.228636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1272/2544] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:02.229470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927dc8fa-f281-45fb-adac-dae427b7c820 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.239570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927dc8fa-f281-45fb-adac-dae427b7c820 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.244909 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722ab73-fd42-4659-93b8-3d614734aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.245523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ab73-fd42-4659-93b8-3d614734aee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.245809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ab73-fd42-4659-93b8-3d614734aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.246084 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ab73-fd42-4659-93b8-3d614734aee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.246733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722ab73-fd42-4659-93b8-3d614734aee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.248371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722ab73-fd42-4659-93b8-3d614734aee7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.264596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1273/2545] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:02.269148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722ab73-fd42-4659-93b8-3d614734aee7 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.278029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1273/2546] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:02.280876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722ab73-fd42-4659-93b8-3d614734aee7 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.285761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff181d-3025-4bb7-8ff9-a40d76bca43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.286407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff181d-3025-4bb7-8ff9-a40d76bca43b admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.286685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff181d-3025-4bb7-8ff9-a40d76bca43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.287011 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff181d-3025-4bb7-8ff9-a40d76bca43b admin admin] PATH_INFO: `/v3/projects/9809d8263f50455c9b4722966633090f/users/ae40a5f09efd452689c9e6e5cdce315f/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.290303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7933041-450b-435d-9885-90d52de41f0f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.291741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f7933041-450b-435d-9885-90d52de41f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:02.303514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff181d-3025-4bb7-8ff9-a40d76bca43b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9809d8263f50455c9b4722966633090f, user_id=ae40a5f09efd452689c9e6e5cdce315f, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.306472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff181d-3025-4bb7-8ff9-a40d76bca43b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.323535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1274/2547] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:02 2022] PUT /identity/v3/projects/9809d8263f50455c9b4722966633090f/users/ae40a5f09efd452689c9e6e5cdce315f/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:02.336305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69433a93-d9df-43f4-9f37-4816f8c91e2f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.336587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69433a93-d9df-43f4-9f37-4816f8c91e2f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.336856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69433a93-d9df-43f4-9f37-4816f8c91e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.362211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:02.365612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-69433a93-d9df-43f4-9f37-4816f8c91e2f None None] MFA Rules not processed for user `ae40a5f09efd452689c9e6e5cdce315f`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:02.366107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7933041-450b-435d-9885-90d52de41f0f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.368765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-69433a93-d9df-43f4-9f37-4816f8c91e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:02.386269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:02.432434 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.433211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.433643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.434784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.479837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1275/2548] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:02.484478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69433a93-d9df-43f4-9f37-4816f8c91e2f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.511415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.512873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.524313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1274/2549] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:06:02.575487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69433a93-d9df-43f4-9f37-4816f8c91e2f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.580505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e31c1-3bbf-4690-8f32-a265fe0314c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.581227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e31c1-3bbf-4690-8f32-a265fe0314c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.581577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e31c1-3bbf-4690-8f32-a265fe0314c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.581917 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e31c1-3bbf-4690-8f32-a265fe0314c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.582744 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e31c1-3bbf-4690-8f32-a265fe0314c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.584922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e31c1-3bbf-4690-8f32-a265fe0314c4 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.619851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.621452 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:02.622930 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1276/2550] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:02 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:02.625478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e31c1-3bbf-4690-8f32-a265fe0314c4 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.634268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b037-bcdb-419a-b099-067cb1af0417 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.687304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a07d6-367e-4514-b4f0-dd7174f370f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.687919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a07d6-367e-4514-b4f0-dd7174f370f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.688086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e31c1-3bbf-4690-8f32-a265fe0314c4 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.688193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a07d6-367e-4514-b4f0-dd7174f370f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.688459 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a07d6-367e-4514-b4f0-dd7174f370f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.692819 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae096d62-23f6-4637-9fb6-2f631abe8a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.693437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae096d62-23f6-4637-9fb6-2f631abe8a0a admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.693722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae096d62-23f6-4637-9fb6-2f631abe8a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.693993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae096d62-23f6-4637-9fb6-2f631abe8a0a admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.694615 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae096d62-23f6-4637-9fb6-2f631abe8a0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.697060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae096d62-23f6-4637-9fb6-2f631abe8a0a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.723758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1277/2551] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:02 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:02.728545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae096d62-23f6-4637-9fb6-2f631abe8a0a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.739399 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae096d62-23f6-4637-9fb6-2f631abe8a0a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.745411 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fcdbd-1117-4bcc-9635-6ed541e8f9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.753757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fcdbd-1117-4bcc-9635-6ed541e8f9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.754034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fcdbd-1117-4bcc-9635-6ed541e8f9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.754316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fcdbd-1117-4bcc-9635-6ed541e8f9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.755657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fcdbd-1117-4bcc-9635-6ed541e8f9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.758093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fcdbd-1117-4bcc-9635-6ed541e8f9e4 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.762985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a07d6-367e-4514-b4f0-dd7174f370f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.764584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a07d6-367e-4514-b4f0-dd7174f370f5 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.770683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1278/2552] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:02.775266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fcdbd-1117-4bcc-9635-6ed541e8f9e4 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.776272 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1275/2553] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:06:02.789957 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fcdbd-1117-4bcc-9635-6ed541e8f9e4 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.796689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7974c-972e-48c1-8a72-c64b4d9a154e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.797602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7974c-972e-48c1-8a72-c64b4d9a154e admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.798118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7974c-972e-48c1-8a72-c64b4d9a154e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.798712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7974c-972e-48c1-8a72-c64b4d9a154e admin admin] PATH_INFO: `/v3/projects/efc76757c8f2466d914be34ae7c4e575/users/dbe7b083711744feb79304989173ad49/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.802148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:02.803328 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133389d-7814-49cf-a11f-b346d854fb83 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.803465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133389d-7814-49cf-a11f-b346d854fb83 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.803767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133389d-7814-49cf-a11f-b346d854fb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.818274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7974c-972e-48c1-8a72-c64b4d9a154e admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc76757c8f2466d914be34ae7c4e575, user_id=dbe7b083711744feb79304989173ad49, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.821513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7974c-972e-48c1-8a72-c64b4d9a154e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.826960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9133389d-7814-49cf-a11f-b346d854fb83 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:02.828441 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9133389d-7814-49cf-a11f-b346d854fb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:02.841590 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1279/2554] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:02 2022] PUT /identity/v3/projects/efc76757c8f2466d914be34ae7c4e575/users/dbe7b083711744feb79304989173ad49/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:02.845673 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7974c-972e-48c1-8a72-c64b4d9a154e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.862762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:02.921101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1276/2555] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:06:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:02.943555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133389d-7814-49cf-a11f-b346d854fb83 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:02.947563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9133389d-7814-49cf-a11f-b346d854fb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:02.952774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7974c-972e-48c1-8a72-c64b4d9a154e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:02.968005 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474031d-53d9-4077-b16a-906d8d4c61ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:02.970147 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474031d-53d9-4077-b16a-906d8d4c61ad admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:02.970861 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474031d-53d9-4077-b16a-906d8d4c61ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:02.971167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474031d-53d9-4077-b16a-906d8d4c61ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:02.974108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474031d-53d9-4077-b16a-906d8d4c61ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:02.977500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474031d-53d9-4077-b16a-906d8d4c61ad admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:02.999681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1280/2556] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:03.004289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474031d-53d9-4077-b16a-906d8d4c61ad admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:03.018203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474031d-53d9-4077-b16a-906d8d4c61ad admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:03.024461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b18638-45f3-497d-b7c7-c8f590c3365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:03.025118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18638-45f3-497d-b7c7-c8f590c3365a admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:03.025405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18638-45f3-497d-b7c7-c8f590c3365a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:03.026115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18638-45f3-497d-b7c7-c8f590c3365a admin admin] PATH_INFO: `/v3/projects/efc76757c8f2466d914be34ae7c4e575/users/dbe7b083711744feb79304989173ad49/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:03.041494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b18638-45f3-497d-b7c7-c8f590c3365a admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc76757c8f2466d914be34ae7c4e575, user_id=dbe7b083711744feb79304989173ad49, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:03.044684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b18638-45f3-497d-b7c7-c8f590c3365a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:03.057175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1281/2557] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:03 2022] PUT /identity/v3/projects/efc76757c8f2466d914be34ae7c4e575/users/dbe7b083711744feb79304989173ad49/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:03.067694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80178ba4-4da2-4094-a28d-54990901effc None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:03.068222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80178ba4-4da2-4094-a28d-54990901effc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:03.068624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80178ba4-4da2-4094-a28d-54990901effc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:03.103614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-80178ba4-4da2-4094-a28d-54990901effc None None] MFA Rules not processed for user `dbe7b083711744feb79304989173ad49`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:03.103849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133389d-7814-49cf-a11f-b346d854fb83 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:03.105065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-80178ba4-4da2-4094-a28d-54990901effc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:03.178182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a2979-445c-4d09-8471-5b82736669a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:03.180351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a2979-445c-4d09-8471-5b82736669a3 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:03.180898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a2979-445c-4d09-8471-5b82736669a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:03.181315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a2979-445c-4d09-8471-5b82736669a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:03.200192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1282/2558] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:03.211868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80178ba4-4da2-4094-a28d-54990901effc None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:03.253147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a2979-445c-4d09-8471-5b82736669a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:03.254702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a2979-445c-4d09-8471-5b82736669a3 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:03.262826 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1277/2559] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:06:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:06:03.270807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80178ba4-4da2-4094-a28d-54990901effc None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:03.275078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e5c05-d0ea-4205-aee7-742f6fd28fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:03.275779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e5c05-d0ea-4205-aee7-742f6fd28fac service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:03.275969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e5c05-d0ea-4205-aee7-742f6fd28fac service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:03.276227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e5c05-d0ea-4205-aee7-742f6fd28fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:03.282204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:03.287252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e5c05-d0ea-4205-aee7-742f6fd28fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:03.289921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e5c05-d0ea-4205-aee7-742f6fd28fac service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:03.300369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1283/2560] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:06:03.990761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a2979-445c-4d09-8471-5b82736669a3 service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:04.002784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a2979-445c-4d09-8471-5b82736669a3 service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:04.007455 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23ec20-f9b5-494e-9c69-96a56e3b9b31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:04.008612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23ec20-f9b5-494e-9c69-96a56e3b9b31 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:04.009115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23ec20-f9b5-494e-9c69-96a56e3b9b31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:04.009464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23ec20-f9b5-494e-9c69-96a56e3b9b31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:04.024606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:04.091484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23ec20-f9b5-494e-9c69-96a56e3b9b31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:04.093249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23ec20-f9b5-494e-9c69-96a56e3b9b31 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:04.103882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1278/2561] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:10.497816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e5c05-d0ea-4205-aee7-742f6fd28fac service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:10.509538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e5c05-d0ea-4205-aee7-742f6fd28fac service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:10.513980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e1819-e5b7-4876-8feb-45f95d71645f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:10.514837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e1819-e5b7-4876-8feb-45f95d71645f service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:10.515679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e1819-e5b7-4876-8feb-45f95d71645f service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:10.516175 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e1819-e5b7-4876-8feb-45f95d71645f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:10.576524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e1819-e5b7-4876-8feb-45f95d71645f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:10.578243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e1819-e5b7-4876-8feb-45f95d71645f service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:10.588477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1284/2562] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:13.010120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23ec20-f9b5-494e-9c69-96a56e3b9b31 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:13.071091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23ec20-f9b5-494e-9c69-96a56e3b9b31 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:13.076025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:13.076645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:13.076923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:13.077191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] PATH_INFO: `/v3/users/74efc33a0d444dcc9075dcfdb1f4cdce` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:13.078291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74efc33a0d444dcc9075dcfdb1f4cdce)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:13.080216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:13.125039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74efc33a0d444dcc9075dcfdb1f4cdce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.131874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74efc33a0d444dcc9075dcfdb1f4cdce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.132516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74efc33a0d444dcc9075dcfdb1f4cdce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.133109 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74efc33a0d444dcc9075dcfdb1f4cdce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.140573 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74efc33a0d444dcc9075dcfdb1f4cdce'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.157820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1279/2563] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:13 2022] DELETE /identity/v3/users/74efc33a0d444dcc9075dcfdb1f4cdce => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:13.162252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944f028-6bca-4598-90fc-2cf59e0fb437 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:13.162755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944f028-6bca-4598-90fc-2cf59e0fb437 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:13.162869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944f028-6bca-4598-90fc-2cf59e0fb437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:13.181556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a944f028-6bca-4598-90fc-2cf59e0fb437 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:13.182731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a944f028-6bca-4598-90fc-2cf59e0fb437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:13.239649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1285/2564] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:13.249516 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:13.250936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:13.286260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:13.309374 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1848b65-b970-4076-97b3-4bedeae24297 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:13.313106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b84efb-14bd-4cff-9e2d-8936f85b59fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:13.313708 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b84efb-14bd-4cff-9e2d-8936f85b59fd service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:13.313992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b84efb-14bd-4cff-9e2d-8936f85b59fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:13.314262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b84efb-14bd-4cff-9e2d-8936f85b59fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:13.323816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b84efb-14bd-4cff-9e2d-8936f85b59fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:13.325243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b84efb-14bd-4cff-9e2d-8936f85b59fd service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:13.334687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1280/2565] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:13.603785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944f028-6bca-4598-90fc-2cf59e0fb437 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:13.606108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a944f028-6bca-4598-90fc-2cf59e0fb437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:13.624769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944f028-6bca-4598-90fc-2cf59e0fb437 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:13.629921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:13.630558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:13.630838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:13.631146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] PATH_INFO: `/v3/projects/dd6bce1a0b164725b24466255767c8c5` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:13.632390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6bce1a0b164725b24466255767c8c5)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:13.634953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:13.686015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6bce1a0b164725b24466255767c8c5', 'request_id': 'req-f21454ce-de12-448a-ab7c-434523d77531'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.693513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6bce1a0b164725b24466255767c8c5', 'request_id': 'req-f21454ce-de12-448a-ab7c-434523d77531'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.700616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6bce1a0b164725b24466255767c8c5', 'request_id': 'req-f21454ce-de12-448a-ab7c-434523d77531'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.714452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] The token cache is being invalidate because project dd6bce1a0b164725b24466255767c8c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:13.714967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.715885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.718559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1286/2566] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:13 2022] DELETE /identity/v3/projects/dd6bce1a0b164725b24466255767c8c5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:13.723803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b84efb-14bd-4cff-9e2d-8936f85b59fd service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:13.725849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:13.727322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:13.735190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b84efb-14bd-4cff-9e2d-8936f85b59fd service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:13.746108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:13.796185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21454ce-de12-448a-ab7c-434523d77531 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:13.798729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:13.799501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:13.799860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:13.800202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] PATH_INFO: `/v3/users/ae40a5f09efd452689c9e6e5cdce315f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:13.801163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:13.801445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae40a5f09efd452689c9e6e5cdce315f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:13.801793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:13.802079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:13.802357 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] PATH_INFO: `/v3/users/b7ba9104ffd94389b72ad9aed27dc3eb` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:13.803505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ba9104ffd94389b72ad9aed27dc3eb)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:13.803636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:13.805455 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:13.852778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae40a5f09efd452689c9e6e5cdce315f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.855543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ba9104ffd94389b72ad9aed27dc3eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.860057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae40a5f09efd452689c9e6e5cdce315f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.860741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae40a5f09efd452689c9e6e5cdce315f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.861336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae40a5f09efd452689c9e6e5cdce315f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.862581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ba9104ffd94389b72ad9aed27dc3eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.863444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ba9104ffd94389b72ad9aed27dc3eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.864097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ba9104ffd94389b72ad9aed27dc3eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.869778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-32761020-c415-4944-8eba-fac5ff305031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae40a5f09efd452689c9e6e5cdce315f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.871879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ba9104ffd94389b72ad9aed27dc3eb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:13.891645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1281/2567] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:13 2022] DELETE /identity/v3/users/ae40a5f09efd452689c9e6e5cdce315f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:13.896263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1287/2568] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:13 2022] DELETE /identity/v3/users/b7ba9104ffd94389b72ad9aed27dc3eb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:13.901122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be09e6-c1df-4a6f-810c-0ea6d5f9c538 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:13.901416 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be09e6-c1df-4a6f-810c-0ea6d5f9c538 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:13.901701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be09e6-c1df-4a6f-810c-0ea6d5f9c538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:13.922544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-56be09e6-c1df-4a6f-810c-0ea6d5f9c538 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:13.923831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-56be09e6-c1df-4a6f-810c-0ea6d5f9c538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:13.974681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:13.988458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1282/2569] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:13.998260 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:13.999772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:14.058339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a352001-4ba7-4422-bd80-8525b89a1271 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:14.061923 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:14.062526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:14.062804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:14.063115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:14.072622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:14.074051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:14.084377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1288/2570] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:14.302805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be09e6-c1df-4a6f-810c-0ea6d5f9c538 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:14.304365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-56be09e6-c1df-4a6f-810c-0ea6d5f9c538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:14.320540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be09e6-c1df-4a6f-810c-0ea6d5f9c538 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:14.325847 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:14.326618 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:14.326995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:14.327609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] PATH_INFO: `/v3/projects/9809d8263f50455c9b4722966633090f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:14.328647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9809d8263f50455c9b4722966633090f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:14.332936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:14.384481 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9809d8263f50455c9b4722966633090f', 'request_id': 'req-59d7f1dc-9e03-416c-81aa-28731825b714'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.393197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9809d8263f50455c9b4722966633090f', 'request_id': 'req-59d7f1dc-9e03-416c-81aa-28731825b714'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.398397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:14.399963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:14.404872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9809d8263f50455c9b4722966633090f', 'request_id': 'req-59d7f1dc-9e03-416c-81aa-28731825b714'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.425414 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] The token cache is being invalidate because project 9809d8263f50455c9b4722966633090f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:14.425806 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.426740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.428433 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1283/2571] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:14 2022] DELETE /identity/v3/projects/9809d8263f50455c9b4722966633090f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:14.431342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ad326-db4b-40b5-9079-ef64a3da0445 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:14.433297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:14.435465 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:14.471558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:14.501591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:14.502137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:14.502408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:14.502681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] PATH_INFO: `/v3/projects/97be096875584107a464610810b6be53` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:14.504101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=97be096875584107a464610810b6be53)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:14.506052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:14.521216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7f1dc-9e03-416c-81aa-28731825b714 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:14.526070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:14.526704 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:14.527019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:14.527290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] PATH_INFO: `/v3/users/dbe7b083711744feb79304989173ad49` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:14.528418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbe7b083711744feb79304989173ad49)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:14.530388 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:14.563386 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97be096875584107a464610810b6be53', 'request_id': 'req-4af9c4e4-1927-40ef-ab76-84328e00ef3c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.574538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97be096875584107a464610810b6be53', 'request_id': 'req-4af9c4e4-1927-40ef-ab76-84328e00ef3c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.583204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97be096875584107a464610810b6be53', 'request_id': 'req-4af9c4e4-1927-40ef-ab76-84328e00ef3c'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.596964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] The token cache is being invalidate because project 97be096875584107a464610810b6be53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:14.597344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.597992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.599837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1289/2572] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:14 2022] DELETE /identity/v3/projects/97be096875584107a464610810b6be53 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:14.604292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:14.605734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:14.674720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c4e4-1927-40ef-ab76-84328e00ef3c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:14.679436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:14.680065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:14.680339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:14.681198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] PATH_INFO: `/v3/users/0af6f42a8786493f94565159cdb30213` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:14.682331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af6f42a8786493f94565159cdb30213)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:14.684315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:14.705492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbe7b083711744feb79304989173ad49'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.713244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbe7b083711744feb79304989173ad49'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.713939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbe7b083711744feb79304989173ad49'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.714665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbe7b083711744feb79304989173ad49'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.722152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbe7b083711744feb79304989173ad49'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.731718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af6f42a8786493f94565159cdb30213'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.737164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:14.738431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af6f42a8786493f94565159cdb30213'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.739154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af6f42a8786493f94565159cdb30213'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.739804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af6f42a8786493f94565159cdb30213'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.744964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1284/2573] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:14 2022] DELETE /identity/v3/users/dbe7b083711744feb79304989173ad49 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:14.752215 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af6f42a8786493f94565159cdb30213'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:14.758185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:14.776544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1290/2574] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:14 2022] DELETE /identity/v3/users/0af6f42a8786493f94565159cdb30213 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:15.042919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.044161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:15.068416 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.069839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:15.144017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3b37e-9532-42f5-8c8c-9b79def8f515 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.187733 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5b9db-6aea-4c62-bce3-b5214a45441c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.195385 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.196032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.196382 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.196664 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] PATH_INFO: `/v3/projects/4a44d82f6a284d95ae40914309cda99c` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.197994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a44d82f6a284d95ae40914309cda99c)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.200103 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.234819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.235990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.236474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.236799 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] PATH_INFO: `/v3/projects/efc76757c8f2466d914be34ae7c4e575` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.237958 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc76757c8f2466d914be34ae7c4e575)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.239909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.257480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a44d82f6a284d95ae40914309cda99c', 'request_id': 'req-dc089ff7-05bc-4fcc-9999-7a2053792aaa'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.267210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a44d82f6a284d95ae40914309cda99c', 'request_id': 'req-dc089ff7-05bc-4fcc-9999-7a2053792aaa'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.276186 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a44d82f6a284d95ae40914309cda99c', 'request_id': 'req-dc089ff7-05bc-4fcc-9999-7a2053792aaa'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.287532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] The token cache is being invalidate because project 4a44d82f6a284d95ae40914309cda99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:15.287831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.288496 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-dc089ff7-05bc-4fcc-9999-7a2053792aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.290956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1291/2575] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:15 2022] DELETE /identity/v3/projects/4a44d82f6a284d95ae40914309cda99c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:15.295417 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6afb6-b4c1-4d9e-b45d-71a276ae140b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.295901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6afb6-b4c1-4d9e-b45d-71a276ae140b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.296032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6afb6-b4c1-4d9e-b45d-71a276ae140b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.297161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc76757c8f2466d914be34ae7c4e575', 'request_id': 'req-e853928e-10a1-4804-8ca2-bf5439d060e5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.315912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-eab6afb6-b4c1-4d9e-b45d-71a276ae140b None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:15.317109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-eab6afb6-b4c1-4d9e-b45d-71a276ae140b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:15.322773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc76757c8f2466d914be34ae7c4e575', 'request_id': 'req-e853928e-10a1-4804-8ca2-bf5439d060e5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.329601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc76757c8f2466d914be34ae7c4e575', 'request_id': 'req-e853928e-10a1-4804-8ca2-bf5439d060e5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.341776 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] The token cache is being invalidate because project efc76757c8f2466d914be34ae7c4e575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:15.342067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.342701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e853928e-10a1-4804-8ca2-bf5439d060e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:15.344212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1285/2576] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:15 2022] DELETE /identity/v3/projects/efc76757c8f2466d914be34ae7c4e575 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:15.349860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb54c-8ee5-4fc0-b304-d34e63aa9e2a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.350141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb54c-8ee5-4fc0-b304-d34e63aa9e2a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.350415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb54c-8ee5-4fc0-b304-d34e63aa9e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.370584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fb0eb54c-8ee5-4fc0-b304-d34e63aa9e2a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:15.371822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fb0eb54c-8ee5-4fc0-b304-d34e63aa9e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:15.383490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1292/2577] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:15.391474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0ea7-97bb-4541-a899-3f9a6e021c62 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.391762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0ea7-97bb-4541-a899-3f9a6e021c62 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.392399 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d0ea7-97bb-4541-a899-3f9a6e021c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.409628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:15.411958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-542d0ea7-97bb-4541-a899-3f9a6e021c62 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:15.413150 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-542d0ea7-97bb-4541-a899-3f9a6e021c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:15.422226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1293/2578] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:15.426850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d0ea7-97bb-4541-a899-3f9a6e021c62 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.436380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d0ea7-97bb-4541-a899-3f9a6e021c62 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.440912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.441495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.441773 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.442042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.442676 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:15.443009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.443396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1286/2579] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:15.444589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.451248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:15.453014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5329392-8aa5-4293-9f3f-e9325cc6afb1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.453303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5329392-8aa5-4293-9f3f-e9325cc6afb1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.453622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5329392-8aa5-4293-9f3f-e9325cc6afb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.456610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1294/2580] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:06:15.460604 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.472023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a869e3-2438-470c-af02-7ba46d160bbc admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.475981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c5329392-8aa5-4293-9f3f-e9325cc6afb1 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:15.476602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18baf-5f68-4f92-8a3f-993c70b5ec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.477160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c5329392-8aa5-4293-9f3f-e9325cc6afb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:15.477268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18baf-5f68-4f92-8a3f-993c70b5ec95 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.477997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18baf-5f68-4f92-8a3f-993c70b5ec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.478270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18baf-5f68-4f92-8a3f-993c70b5ec95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.478919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a18baf-5f68-4f92-8a3f-993c70b5ec95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.480835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a18baf-5f68-4f92-8a3f-993c70b5ec95 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.485369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1287/2581] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:15.489519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5329392-8aa5-4293-9f3f-e9325cc6afb1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.498731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5329392-8aa5-4293-9f3f-e9325cc6afb1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.504248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.504928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.505282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.505623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.506348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:15.506722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.511389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1295/2582] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:15.511537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.512379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18baf-5f68-4f92-8a3f-993c70b5ec95 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.520226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1288/2583] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:06:15.524511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.534053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc079a4b-c8f3-43c5-8d41-f58c9d53348f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.574155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18baf-5f68-4f92-8a3f-993c70b5ec95 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.579021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389f629-8d4b-4f74-9b70-f8b86460a6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.579621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389f629-8d4b-4f74-9b70-f8b86460a6ec admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.579919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389f629-8d4b-4f74-9b70-f8b86460a6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.580188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389f629-8d4b-4f74-9b70-f8b86460a6ec admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.580808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389f629-8d4b-4f74-9b70-f8b86460a6ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.582926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389f629-8d4b-4f74-9b70-f8b86460a6ec admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.591243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ad401-a197-4856-9109-6bdc2391ab22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.591849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ad401-a197-4856-9109-6bdc2391ab22 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.592128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ad401-a197-4856-9109-6bdc2391ab22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.592401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ad401-a197-4856-9109-6bdc2391ab22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.593019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ad401-a197-4856-9109-6bdc2391ab22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.594955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ad401-a197-4856-9109-6bdc2391ab22 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.604174 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1296/2584] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:15.608198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389f629-8d4b-4f74-9b70-f8b86460a6ec admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.620119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389f629-8d4b-4f74-9b70-f8b86460a6ec admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.623587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1289/2585] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/projects => generated 407 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:15.624747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ffed8-539f-4e4e-8657-3a7d6b8be8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.625338 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ffed8-539f-4e4e-8657-3a7d6b8be8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.625620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ffed8-539f-4e4e-8657-3a7d6b8be8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.625890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ffed8-539f-4e4e-8657-3a7d6b8be8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.626519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ffed8-539f-4e4e-8657-3a7d6b8be8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.627587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ad401-a197-4856-9109-6bdc2391ab22 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.628159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ffed8-539f-4e4e-8657-3a7d6b8be8ab admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.637970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1297/2586] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:15.641704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ffed8-539f-4e4e-8657-3a7d6b8be8ab admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.651132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ffed8-539f-4e4e-8657-3a7d6b8be8ab admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.690731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ad401-a197-4856-9109-6bdc2391ab22 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.695678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2b27e-c637-4996-8db2-ab18dc9d169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.696272 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b27e-c637-4996-8db2-ab18dc9d169b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.696546 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b27e-c637-4996-8db2-ab18dc9d169b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.697049 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b27e-c637-4996-8db2-ab18dc9d169b admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.697424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2b27e-c637-4996-8db2-ab18dc9d169b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.699355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2b27e-c637-4996-8db2-ab18dc9d169b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.709409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bc0ae-cc20-48c9-90d9-a305eb2196a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.710339 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bc0ae-cc20-48c9-90d9-a305eb2196a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.710622 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bc0ae-cc20-48c9-90d9-a305eb2196a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.711118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bc0ae-cc20-48c9-90d9-a305eb2196a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.712335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bc0ae-cc20-48c9-90d9-a305eb2196a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.713915 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bc0ae-cc20-48c9-90d9-a305eb2196a6 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.721222 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1290/2587] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/users => generated 304 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:15.727056 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1298/2588] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:15.727221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2b27e-c637-4996-8db2-ab18dc9d169b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.728589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bc0ae-cc20-48c9-90d9-a305eb2196a6 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.735853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2b27e-c637-4996-8db2-ab18dc9d169b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.738116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bc0ae-cc20-48c9-90d9-a305eb2196a6 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.740384 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9e30c-cf1a-496a-8375-f80e691bfcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.740964 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9e30c-cf1a-496a-8375-f80e691bfcc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.741270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9e30c-cf1a-496a-8375-f80e691bfcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.741540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9e30c-cf1a-496a-8375-f80e691bfcc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.742163 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9e30c-cf1a-496a-8375-f80e691bfcc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.742698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb5f7f-b964-46c5-9509-157a282375ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.743401 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb5f7f-b964-46c5-9509-157a282375ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.743678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb5f7f-b964-46c5-9509-157a282375ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.743907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9e30c-cf1a-496a-8375-f80e691bfcc1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.744012 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb5f7f-b964-46c5-9509-157a282375ea admin admin] PATH_INFO: `/v3/projects/10ef065e40774935b7ee06648edf5f46/users/0d4dbd7cc9c7411ab13ba708b28337dd/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.753787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1291/2589] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:15.757817 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9e30c-cf1a-496a-8375-f80e691bfcc1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.758446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb5f7f-b964-46c5-9509-157a282375ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ef065e40774935b7ee06648edf5f46, user_id=0d4dbd7cc9c7411ab13ba708b28337dd, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.761107 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb5f7f-b964-46c5-9509-157a282375ea admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.767857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9e30c-cf1a-496a-8375-f80e691bfcc1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.772748 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3f38a-84d5-471b-893b-6693c14ed764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.773353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3f38a-84d5-471b-893b-6693c14ed764 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.773755 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3f38a-84d5-471b-893b-6693c14ed764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.774044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3f38a-84d5-471b-893b-6693c14ed764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.775273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3f38a-84d5-471b-893b-6693c14ed764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.775503 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1299/2590] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:15 2022] PUT /identity/v3/projects/10ef065e40774935b7ee06648edf5f46/users/0d4dbd7cc9c7411ab13ba708b28337dd/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:15.776978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3f38a-84d5-471b-893b-6693c14ed764 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.785477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b47f-6b48-43e5-99d2-9c4512909827 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.785762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b47f-6b48-43e5-99d2-9c4512909827 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.786034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b47f-6b48-43e5-99d2-9c4512909827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.788622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1292/2591] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:15.793008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3f38a-84d5-471b-893b-6693c14ed764 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.802648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3f38a-84d5-471b-893b-6693c14ed764 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.808074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:15.817042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e739b47f-6b48-43e5-99d2-9c4512909827 None None] MFA Rules not processed for user `0d4dbd7cc9c7411ab13ba708b28337dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:15.818252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e739b47f-6b48-43e5-99d2-9c4512909827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:15.830750 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:15.864446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d2b84-f7b8-4e93-ac6e-3bee1bc34aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.865110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d2b84-f7b8-4e93-ac6e-3bee1bc34aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.865386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d2b84-f7b8-4e93-ac6e-3bee1bc34aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.865655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d2b84-f7b8-4e93-ac6e-3bee1bc34aec admin admin] PATH_INFO: `/v3/projects/c613b720b64e457caa2a34907868c098/users/56ba7f6377934f10b1143004eda73f7f/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.891375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d2b84-f7b8-4e93-ac6e-3bee1bc34aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c613b720b64e457caa2a34907868c098, user_id=56ba7f6377934f10b1143004eda73f7f, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.894121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d2b84-f7b8-4e93-ac6e-3bee1bc34aec admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:15.895201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1300/2592] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:15.902311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739b47f-6b48-43e5-99d2-9c4512909827 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:15.908905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1293/2593] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:15 2022] PUT /identity/v3/projects/c613b720b64e457caa2a34907868c098/users/56ba7f6377934f10b1143004eda73f7f/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:15.913426 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739b47f-6b48-43e5-99d2-9c4512909827 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:15.920651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304ca6e-c1b1-4ed6-8ab8-8c6eab375ecd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.920932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304ca6e-c1b1-4ed6-8ab8-8c6eab375ecd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.921202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304ca6e-c1b1-4ed6-8ab8-8c6eab375ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.947921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7304ca6e-c1b1-4ed6-8ab8-8c6eab375ecd None None] MFA Rules not processed for user `56ba7f6377934f10b1143004eda73f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:15.949112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7304ca6e-c1b1-4ed6-8ab8-8c6eab375ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:15.968671 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee004efd-2288-43f7-90f8-896618e7b7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:15.969274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee004efd-2288-43f7-90f8-896618e7b7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:15.969544 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee004efd-2288-43f7-90f8-896618e7b7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:15.969807 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee004efd-2288-43f7-90f8-896618e7b7c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:15.970419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee004efd-2288-43f7-90f8-896618e7b7c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:15.972321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee004efd-2288-43f7-90f8-896618e7b7c7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.000085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1301/2594] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:16.003980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee004efd-2288-43f7-90f8-896618e7b7c7 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.015260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1294/2595] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:16.019609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304ca6e-c1b1-4ed6-8ab8-8c6eab375ecd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.029860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304ca6e-c1b1-4ed6-8ab8-8c6eab375ecd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.066928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee004efd-2288-43f7-90f8-896618e7b7c7 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.072685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7133384-ad6c-43c2-bfeb-4791ab31e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.073279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133384-ad6c-43c2-bfeb-4791ab31e368 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.073553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133384-ad6c-43c2-bfeb-4791ab31e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.074091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133384-ad6c-43c2-bfeb-4791ab31e368 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.074437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7133384-ad6c-43c2-bfeb-4791ab31e368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.076358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7133384-ad6c-43c2-bfeb-4791ab31e368 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.085160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f387a-8d4a-46e8-9731-35f7c55e8afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.085697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f387a-8d4a-46e8-9731-35f7c55e8afe admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.085977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f387a-8d4a-46e8-9731-35f7c55e8afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.086243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f387a-8d4a-46e8-9731-35f7c55e8afe admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.086879 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f387a-8d4a-46e8-9731-35f7c55e8afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.088855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f387a-8d4a-46e8-9731-35f7c55e8afe admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.098705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1302/2596] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:16 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:16.103104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7133384-ad6c-43c2-bfeb-4791ab31e368 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.112956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7133384-ad6c-43c2-bfeb-4791ab31e368 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.118691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfde0f-0ab2-4df8-8a0e-77a13b2a4e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.119497 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfde0f-0ab2-4df8-8a0e-77a13b2a4e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.119850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfde0f-0ab2-4df8-8a0e-77a13b2a4e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.120069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfde0f-0ab2-4df8-8a0e-77a13b2a4e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.120193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1295/2597] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:16 2022] POST /identity/v3/projects => generated 409 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:16.120745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfde0f-0ab2-4df8-8a0e-77a13b2a4e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.122383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfde0f-0ab2-4df8-8a0e-77a13b2a4e53 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.124335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f387a-8d4a-46e8-9731-35f7c55e8afe admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.133344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:16.133524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1303/2598] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:16.137592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfde0f-0ab2-4df8-8a0e-77a13b2a4e53 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.146997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfde0f-0ab2-4df8-8a0e-77a13b2a4e53 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.162812 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:16.191697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f387a-8d4a-46e8-9731-35f7c55e8afe admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.196315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a8ff6-2adc-4e2b-8d1a-5199c815863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.196909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a8ff6-2adc-4e2b-8d1a-5199c815863c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.197186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a8ff6-2adc-4e2b-8d1a-5199c815863c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.197458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a8ff6-2adc-4e2b-8d1a-5199c815863c admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.198072 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a8ff6-2adc-4e2b-8d1a-5199c815863c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.200013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a8ff6-2adc-4e2b-8d1a-5199c815863c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.204083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa00ee-598d-4538-b275-a170308e6207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.204725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa00ee-598d-4538-b275-a170308e6207 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.204999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa00ee-598d-4538-b275-a170308e6207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.205264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa00ee-598d-4538-b275-a170308e6207 admin admin] PATH_INFO: `/v3/projects/25f7e016c3f644dc9fb4d04ee290c397/users/4cc108b2108f4670920af96beb2cc753/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.221156 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa00ee-598d-4538-b275-a170308e6207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f7e016c3f644dc9fb4d04ee290c397, user_id=4cc108b2108f4670920af96beb2cc753, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.222352 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1296/2599] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:16 2022] POST /identity/v3/users => generated 311 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:16.223962 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa00ee-598d-4538-b275-a170308e6207 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.226169 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a8ff6-2adc-4e2b-8d1a-5199c815863c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.236487 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a8ff6-2adc-4e2b-8d1a-5199c815863c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.237884 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1304/2600] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:16 2022] PUT /identity/v3/projects/25f7e016c3f644dc9fb4d04ee290c397/users/4cc108b2108f4670920af96beb2cc753/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:16.241711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa00ee-598d-4538-b275-a170308e6207 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.291747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e08bc8-4b16-4898-bf78-cadb51587bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.292350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e08bc8-4b16-4898-bf78-cadb51587bde admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.292622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e08bc8-4b16-4898-bf78-cadb51587bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.292891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e08bc8-4b16-4898-bf78-cadb51587bde admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.293515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e08bc8-4b16-4898-bf78-cadb51587bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.295102 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e08bc8-4b16-4898-bf78-cadb51587bde admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.301356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa00ee-598d-4538-b275-a170308e6207 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.305036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1297/2601] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:16.308977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61be0-8db4-469e-8713-aa162a349c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.308977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61be0-8db4-469e-8713-aa162a349c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.308977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61be0-8db4-469e-8713-aa162a349c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.308977 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61be0-8db4-469e-8713-aa162a349c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.309395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad61be0-8db4-469e-8713-aa162a349c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.309804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e08bc8-4b16-4898-bf78-cadb51587bde admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.310023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad61be0-8db4-469e-8713-aa162a349c3a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.319366 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e08bc8-4b16-4898-bf78-cadb51587bde admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.319835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1305/2602] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:16.324469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cf7a5-be1f-407f-a4ca-2b6c3c87b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.324946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61be0-8db4-469e-8713-aa162a349c3a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.325071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cf7a5-be1f-407f-a4ca-2b6c3c87b5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.325357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cf7a5-be1f-407f-a4ca-2b6c3c87b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.325629 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cf7a5-be1f-407f-a4ca-2b6c3c87b5d0 admin admin] PATH_INFO: `/v3/projects/f805c8c652864f64a2a553fed768a756/users/3c058e1a9457425aa05b4d3dd6d2e158/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.335234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61be0-8db4-469e-8713-aa162a349c3a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.340292 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14316b-d3ef-4308-9bc9-c9a0e2838f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.340831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14316b-d3ef-4308-9bc9-c9a0e2838f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.341114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14316b-d3ef-4308-9bc9-c9a0e2838f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.341387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14316b-d3ef-4308-9bc9-c9a0e2838f76 admin admin] PATH_INFO: `/v3/projects/25f7e016c3f644dc9fb4d04ee290c397/users/4cc108b2108f4670920af96beb2cc753/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.343506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14316b-d3ef-4308-9bc9-c9a0e2838f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f7e016c3f644dc9fb4d04ee290c397, user_id=4cc108b2108f4670920af96beb2cc753, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.346440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14316b-d3ef-4308-9bc9-c9a0e2838f76 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.348402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cf7a5-be1f-407f-a4ca-2b6c3c87b5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f805c8c652864f64a2a553fed768a756, user_id=3c058e1a9457425aa05b4d3dd6d2e158, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.352182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cf7a5-be1f-407f-a4ca-2b6c3c87b5d0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.369171 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1306/2603] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:16 2022] PUT /identity/v3/projects/25f7e016c3f644dc9fb4d04ee290c397/users/4cc108b2108f4670920af96beb2cc753/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:16.374440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2245e-2551-4e55-9fc1-15dcc3144a34 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.374959 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2245e-2551-4e55-9fc1-15dcc3144a34 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.375307 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2245e-2551-4e55-9fc1-15dcc3144a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.379317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1298/2604] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:16 2022] PUT /identity/v3/projects/f805c8c652864f64a2a553fed768a756/users/3c058e1a9457425aa05b4d3dd6d2e158/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:16.380403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cf7a5-be1f-407f-a4ca-2b6c3c87b5d0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.425916 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a6a2245e-2551-4e55-9fc1-15dcc3144a34 None None] MFA Rules not processed for user `4cc108b2108f4670920af96beb2cc753`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:16.427172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a6a2245e-2551-4e55-9fc1-15dcc3144a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:16.469496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cf7a5-be1f-407f-a4ca-2b6c3c87b5d0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.474240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8467555-fe69-467a-b936-ed9c6f35b8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.474854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8467555-fe69-467a-b936-ed9c6f35b8ad admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.475166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8467555-fe69-467a-b936-ed9c6f35b8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.475582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8467555-fe69-467a-b936-ed9c6f35b8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.476235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8467555-fe69-467a-b936-ed9c6f35b8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.478606 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8467555-fe69-467a-b936-ed9c6f35b8ad admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.489781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1299/2605] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:06:16.494115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8467555-fe69-467a-b936-ed9c6f35b8ad admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.502632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:16.509960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8467555-fe69-467a-b936-ed9c6f35b8ad admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.521500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe24952-2c12-4672-8ce7-1066c00d03e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.522168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe24952-2c12-4672-8ce7-1066c00d03e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.522446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe24952-2c12-4672-8ce7-1066c00d03e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.523122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe24952-2c12-4672-8ce7-1066c00d03e6 admin admin] PATH_INFO: `/v3/projects/f805c8c652864f64a2a553fed768a756/users/3c058e1a9457425aa05b4d3dd6d2e158/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.524965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1307/2606] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:16.528435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe24952-2c12-4672-8ce7-1066c00d03e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f805c8c652864f64a2a553fed768a756, user_id=3c058e1a9457425aa05b4d3dd6d2e158, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.531247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe24952-2c12-4672-8ce7-1066c00d03e6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.535974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2245e-2551-4e55-9fc1-15dcc3144a34 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.537543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a6a2245e-2551-4e55-9fc1-15dcc3144a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:16.553308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:16.565763 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1300/2607] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:16 2022] PUT /identity/v3/projects/f805c8c652864f64a2a553fed768a756/users/3c058e1a9457425aa05b4d3dd6d2e158/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:16.578828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6cb09-0a7f-40a5-a784-1fcda2b00434 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.579191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6cb09-0a7f-40a5-a784-1fcda2b00434 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.579405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6cb09-0a7f-40a5-a784-1fcda2b00434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.614647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2245e-2551-4e55-9fc1-15dcc3144a34 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.615687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-92c6cb09-0a7f-40a5-a784-1fcda2b00434 None None] MFA Rules not processed for user `3c058e1a9457425aa05b4d3dd6d2e158`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:16.616969 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-92c6cb09-0a7f-40a5-a784-1fcda2b00434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:16.674777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1420b-1261-450e-b773-cc0c7cfadd92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.675543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1420b-1261-450e-b773-cc0c7cfadd92 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.675933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1420b-1261-450e-b773-cc0c7cfadd92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.676266 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1420b-1261-450e-b773-cc0c7cfadd92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.698309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1301/2608] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:16 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:16.709466 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6cb09-0a7f-40a5-a784-1fcda2b00434 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.710985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-92c6cb09-0a7f-40a5-a784-1fcda2b00434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:16.724172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6cb09-0a7f-40a5-a784-1fcda2b00434 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.729274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9625e-8b6e-4e27-854e-d954e213e0b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.730938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9625e-8b6e-4e27-854e-d954e213e0b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.731254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9625e-8b6e-4e27-854e-d954e213e0b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.731520 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9625e-8b6e-4e27-854e-d954e213e0b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:16.738725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1420b-1261-450e-b773-cc0c7cfadd92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.740224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1420b-1261-450e-b773-cc0c7cfadd92 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.741300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9625e-8b6e-4e27-854e-d954e213e0b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:16.742700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9625e-8b6e-4e27-854e-d954e213e0b5 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:16.752427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1308/2609] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:16.754689 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1302/2610] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:16 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:06:16.973726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1420b-1261-450e-b773-cc0c7cfadd92 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:16.982741 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1420b-1261-450e-b773-cc0c7cfadd92 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:16.986535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:16.987164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:16.987442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:16.987714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:17.064157 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:17.065669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:17.080350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1309/2611] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:16 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:18.013068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ad48c-790c-4b69-948f-7d751a311546 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:18.013543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ad48c-790c-4b69-948f-7d751a311546 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:18.013833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ad48c-790c-4b69-948f-7d751a311546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:18.036426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d31ad48c-790c-4b69-948f-7d751a311546 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:18.037995 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d31ad48c-790c-4b69-948f-7d751a311546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:18.109074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1303/2612] 10.222.0.52 () {58 vars in 967 bytes} [Mon Oct 10 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:18.121581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:18.124139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:18.147492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:18.196745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8cbf-982b-41a9-be94-a977275b18d8 service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:18.201790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bb02b-9bd3-4665-a60d-cd0f5e58d559 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:18.202521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bb02b-9bd3-4665-a60d-cd0f5e58d559 service swift] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:18.202798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bb02b-9bd3-4665-a60d-cd0f5e58d559 service swift] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:18.203096 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bb02b-9bd3-4665-a60d-cd0f5e58d559 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:18.211324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bb02b-9bd3-4665-a60d-cd0f5e58d559 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:18.214177 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bb02b-9bd3-4665-a60d-cd0f5e58d559 service swift] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:18.224288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1310/2613] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:06:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:06:22.454025 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ad48c-790c-4b69-948f-7d751a311546 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:22.463730 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ad48c-790c-4b69-948f-7d751a311546 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:22.467207 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117ee0a-ea4d-43f5-885b-1e84a40bcee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:22.467803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ee0a-ea4d-43f5-885b-1e84a40bcee6 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:22.468080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ee0a-ea4d-43f5-885b-1e84a40bcee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:22.468343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ee0a-ea4d-43f5-885b-1e84a40bcee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:22.556860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117ee0a-ea4d-43f5-885b-1e84a40bcee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:22.558282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117ee0a-ea4d-43f5-885b-1e84a40bcee6 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:22.567796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1304/2614] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:28.219804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=23 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:31.702109 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bb02b-9bd3-4665-a60d-cd0f5e58d559 service swift] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:31.762653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bb02b-9bd3-4665-a60d-cd0f5e58d559 service swift] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:31.767221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:31.767840 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:31.768123 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:31.768398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] PATH_INFO: `/v3/users/56ba7f6377934f10b1143004eda73f7f` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:31.769504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56ba7f6377934f10b1143004eda73f7f)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:31.771459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:31.817058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56ba7f6377934f10b1143004eda73f7f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:31.824461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56ba7f6377934f10b1143004eda73f7f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:31.825119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56ba7f6377934f10b1143004eda73f7f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:31.825721 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56ba7f6377934f10b1143004eda73f7f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:31.833543 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56ba7f6377934f10b1143004eda73f7f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:31.852742 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1311/2615] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:31 2022] DELETE /identity/v3/users/56ba7f6377934f10b1143004eda73f7f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:31.860421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec30c8-e826-4c94-bae3-d857117c7c04 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:31.860889 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec30c8-e826-4c94-bae3-d857117c7c04 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:31.861189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec30c8-e826-4c94-bae3-d857117c7c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:31.868477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:31.882082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3eec30c8-e826-4c94-bae3-d857117c7c04 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:31.883872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3eec30c8-e826-4c94-bae3-d857117c7c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:31.947308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1305/2616] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:31.957381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:31.958921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:32.021216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1e2f2-42ea-40e3-b2b8-027cbc8e35d4 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:32.026834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83855-ed35-4505-9ed6-ea6b16d115d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:32.027490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83855-ed35-4505-9ed6-ea6b16d115d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:32.027783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83855-ed35-4505-9ed6-ea6b16d115d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:32.028204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83855-ed35-4505-9ed6-ea6b16d115d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:32.044389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed83855-ed35-4505-9ed6-ea6b16d115d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:32.045821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed83855-ed35-4505-9ed6-ea6b16d115d7 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:32.055844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1312/2617] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:32.318833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec30c8-e826-4c94-bae3-d857117c7c04 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:32.320421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3eec30c8-e826-4c94-bae3-d857117c7c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:32.335765 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec30c8-e826-4c94-bae3-d857117c7c04 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:32.343171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:32.343832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:32.344074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:32.344350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] PATH_INFO: `/v3/projects/c613b720b64e457caa2a34907868c098` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:32.345479 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c613b720b64e457caa2a34907868c098)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:32.347400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:32.395519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c613b720b64e457caa2a34907868c098', 'request_id': 'req-879d76cf-0e23-47cb-aef4-a3f9815b50ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.403572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c613b720b64e457caa2a34907868c098', 'request_id': 'req-879d76cf-0e23-47cb-aef4-a3f9815b50ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.410180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c613b720b64e457caa2a34907868c098', 'request_id': 'req-879d76cf-0e23-47cb-aef4-a3f9815b50ee'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.422082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] The token cache is being invalidate because project c613b720b64e457caa2a34907868c098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:32.422372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.423310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.424818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1306/2618] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:32 2022] DELETE /identity/v3/projects/c613b720b64e457caa2a34907868c098 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:32.428786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83855-ed35-4505-9ed6-ea6b16d115d7 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:32.438315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83855-ed35-4505-9ed6-ea6b16d115d7 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:32.455902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:32.494223 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:32.494839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:32.495148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:32.495420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] PATH_INFO: `/v3/users/3c058e1a9457425aa05b4d3dd6d2e158` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:32.496516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c058e1a9457425aa05b4d3dd6d2e158)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:32.498427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:32.564620 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c058e1a9457425aa05b4d3dd6d2e158'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.572815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c058e1a9457425aa05b4d3dd6d2e158'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.573489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c058e1a9457425aa05b4d3dd6d2e158'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.574097 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c058e1a9457425aa05b4d3dd6d2e158'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.581410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-db28c03e-d9a2-4b30-9be5-f0a1dceeb040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c058e1a9457425aa05b4d3dd6d2e158'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.598717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1313/2619] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:32 2022] DELETE /identity/v3/users/3c058e1a9457425aa05b4d3dd6d2e158 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:32.860633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:32.862110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:32.875008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d76cf-0e23-47cb-aef4-a3f9815b50ee admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:32.931483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:32.932101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:32.932378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:32.932647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] PATH_INFO: `/v3/projects/f805c8c652864f64a2a553fed768a756` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:32.933864 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f805c8c652864f64a2a553fed768a756)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:32.935863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:32.966609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:32.982474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f805c8c652864f64a2a553fed768a756', 'request_id': 'req-cba95259-dcf8-4d86-ba92-4369c42eaa1d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.989378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f805c8c652864f64a2a553fed768a756', 'request_id': 'req-cba95259-dcf8-4d86-ba92-4369c42eaa1d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:32.996098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f805c8c652864f64a2a553fed768a756', 'request_id': 'req-cba95259-dcf8-4d86-ba92-4369c42eaa1d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:33.006070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] The token cache is being invalidate because project f805c8c652864f64a2a553fed768a756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:33.006353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:33.006999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cba95259-dcf8-4d86-ba92-4369c42eaa1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:33.009114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1307/2620] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:32 2022] DELETE /identity/v3/projects/f805c8c652864f64a2a553fed768a756 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:33.013900 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06961edc-bcad-4582-80b5-54f373b6bc0d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.014361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06961edc-bcad-4582-80b5-54f373b6bc0d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.014527 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06961edc-bcad-4582-80b5-54f373b6bc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.033602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-06961edc-bcad-4582-80b5-54f373b6bc0d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:33.034785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-06961edc-bcad-4582-80b5-54f373b6bc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:33.094477 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1314/2621] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:33.102091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d3fb4-6354-4b8c-97f7-3617c62a5f81 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.102379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d3fb4-6354-4b8c-97f7-3617c62a5f81 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.102653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d3fb4-6354-4b8c-97f7-3617c62a5f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.122084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-333d3fb4-6354-4b8c-97f7-3617c62a5f81 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:33.123300 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-333d3fb4-6354-4b8c-97f7-3617c62a5f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:33.130951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1308/2622] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:33.135659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06961edc-bcad-4582-80b5-54f373b6bc0d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.145030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06961edc-bcad-4582-80b5-54f373b6bc0d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.151200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.151805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.152082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.152350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.152993 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:33.153293 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.154905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.166279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1315/2623] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:06:33.170691 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d3fb4-6354-4b8c-97f7-3617c62a5f81 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.181087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d3fb4-6354-4b8c-97f7-3617c62a5f81 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.185669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec596c-b2f4-4bc7-81ea-e3a38c059a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.186253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec596c-b2f4-4bc7-81ea-e3a38c059a57 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.186529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec596c-b2f4-4bc7-81ea-e3a38c059a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.186793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec596c-b2f4-4bc7-81ea-e3a38c059a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.187700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec596c-b2f4-4bc7-81ea-e3a38c059a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.189584 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec596c-b2f4-4bc7-81ea-e3a38c059a57 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.216244 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1309/2624] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:33 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:33.220762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.230790 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef150-ff64-4352-a327-f2ee0960b504 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.257374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:33.288629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678dfd2-ac57-44b7-a35a-89a779aed7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.289224 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678dfd2-ac57-44b7-a35a-89a779aed7ad admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.289547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678dfd2-ac57-44b7-a35a-89a779aed7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.289801 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678dfd2-ac57-44b7-a35a-89a779aed7ad admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.290414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678dfd2-ac57-44b7-a35a-89a779aed7ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.292328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678dfd2-ac57-44b7-a35a-89a779aed7ad admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.313312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1316/2625] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:33 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:33.317570 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec596c-b2f4-4bc7-81ea-e3a38c059a57 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.328140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec596c-b2f4-4bc7-81ea-e3a38c059a57 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.332646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae06da-2bae-46fd-a1f8-ea8296acaf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.333229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae06da-2bae-46fd-a1f8-ea8296acaf05 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.333506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae06da-2bae-46fd-a1f8-ea8296acaf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.333771 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae06da-2bae-46fd-a1f8-ea8296acaf05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.334394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae06da-2bae-46fd-a1f8-ea8296acaf05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.335976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae06da-2bae-46fd-a1f8-ea8296acaf05 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.345607 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1310/2626] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:33.349362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678dfd2-ac57-44b7-a35a-89a779aed7ad admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.359561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678dfd2-ac57-44b7-a35a-89a779aed7ad admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.364093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33212e-67e8-453e-acc5-aea282c0fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.364675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33212e-67e8-453e-acc5-aea282c0fa3c admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.364951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33212e-67e8-453e-acc5-aea282c0fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.365217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33212e-67e8-453e-acc5-aea282c0fa3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.365844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33212e-67e8-453e-acc5-aea282c0fa3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.367452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33212e-67e8-453e-acc5-aea282c0fa3c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.378188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1317/2627] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:33.382103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae06da-2bae-46fd-a1f8-ea8296acaf05 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.391818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae06da-2bae-46fd-a1f8-ea8296acaf05 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.396415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef86f-4ef6-430e-b761-2c169569f8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.397039 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef86f-4ef6-430e-b761-2c169569f8b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.397318 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef86f-4ef6-430e-b761-2c169569f8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.397591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef86f-4ef6-430e-b761-2c169569f8b9 admin admin] PATH_INFO: `/v3/projects/b92feee8f1ee4073abfae09015da1705/users/94ec95f6f4b642abaf6add2ada5fafc5/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.412313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ef86f-4ef6-430e-b761-2c169569f8b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b92feee8f1ee4073abfae09015da1705, user_id=94ec95f6f4b642abaf6add2ada5fafc5, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.415560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ef86f-4ef6-430e-b761-2c169569f8b9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.428478 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1311/2628] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:33 2022] PUT /identity/v3/projects/b92feee8f1ee4073abfae09015da1705/users/94ec95f6f4b642abaf6add2ada5fafc5/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:33.439240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b77ae8-07c6-4b53-8caf-35ec2c419fe4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.439526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b77ae8-07c6-4b53-8caf-35ec2c419fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.439800 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b77ae8-07c6-4b53-8caf-35ec2c419fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.466990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e0b77ae8-07c6-4b53-8caf-35ec2c419fe4 None None] MFA Rules not processed for user `94ec95f6f4b642abaf6add2ada5fafc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:33.468181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e0b77ae8-07c6-4b53-8caf-35ec2c419fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:33.529116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1318/2629] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:33.533426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef86f-4ef6-430e-b761-2c169569f8b9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.601699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef86f-4ef6-430e-b761-2c169569f8b9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.606733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccdee4-575a-4c61-a792-67213f0fa981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.607362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccdee4-575a-4c61-a792-67213f0fa981 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.607641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccdee4-575a-4c61-a792-67213f0fa981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.607912 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccdee4-575a-4c61-a792-67213f0fa981 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.608533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccdee4-575a-4c61-a792-67213f0fa981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.610389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccdee4-575a-4c61-a792-67213f0fa981 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.636511 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1312/2630] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:33 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:33.640625 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b77ae8-07c6-4b53-8caf-35ec2c419fe4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.652656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b77ae8-07c6-4b53-8caf-35ec2c419fe4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.710093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6ff41-db23-49a9-abf4-cf06ed5df71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.710778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ff41-db23-49a9-abf4-cf06ed5df71f admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.711094 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ff41-db23-49a9-abf4-cf06ed5df71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.711876 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ff41-db23-49a9-abf4-cf06ed5df71f admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.712506 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6ff41-db23-49a9-abf4-cf06ed5df71f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.714410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6ff41-db23-49a9-abf4-cf06ed5df71f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.736232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1319/2631] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:33 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:33.740192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccdee4-575a-4c61-a792-67213f0fa981 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.750557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccdee4-575a-4c61-a792-67213f0fa981 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.755153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edecaf-a93f-4983-96db-ecf52ee45f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.755747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edecaf-a93f-4983-96db-ecf52ee45f43 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.756027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edecaf-a93f-4983-96db-ecf52ee45f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.756299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edecaf-a93f-4983-96db-ecf52ee45f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.756935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edecaf-a93f-4983-96db-ecf52ee45f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.758540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edecaf-a93f-4983-96db-ecf52ee45f43 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.769582 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1313/2632] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:33.773642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6ff41-db23-49a9-abf4-cf06ed5df71f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.784353 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6ff41-db23-49a9-abf4-cf06ed5df71f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.788881 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff2f9d-890a-457c-9df0-270003ce9612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.790120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff2f9d-890a-457c-9df0-270003ce9612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.790402 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff2f9d-890a-457c-9df0-270003ce9612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.790670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff2f9d-890a-457c-9df0-270003ce9612 admin admin] PATH_INFO: `/v3/projects/b30b71f082204338aba8410cdf87a5fc/users/2d460337950b45d3825b38c627f2dcaa/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.805085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff2f9d-890a-457c-9df0-270003ce9612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30b71f082204338aba8410cdf87a5fc, user_id=2d460337950b45d3825b38c627f2dcaa, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.807731 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff2f9d-890a-457c-9df0-270003ce9612 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.819786 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1320/2633] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:33 2022] PUT /identity/v3/projects/b30b71f082204338aba8410cdf87a5fc/users/2d460337950b45d3825b38c627f2dcaa/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:33.823723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edecaf-a93f-4983-96db-ecf52ee45f43 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.833201 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edecaf-a93f-4983-96db-ecf52ee45f43 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.857080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:33.889571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fcc46-6fcb-49a8-8f6e-d08925fcbef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.890228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fcc46-6fcb-49a8-8f6e-d08925fcbef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.890437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fcc46-6fcb-49a8-8f6e-d08925fcbef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.890703 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fcc46-6fcb-49a8-8f6e-d08925fcbef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.891364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fcc46-6fcb-49a8-8f6e-d08925fcbef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.892922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fcc46-6fcb-49a8-8f6e-d08925fcbef5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.902966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1314/2634] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:33.907220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff2f9d-890a-457c-9df0-270003ce9612 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:33.912591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:33.919432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff2f9d-890a-457c-9df0-270003ce9612 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:33.924553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:33.925194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.925498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.925772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] PATH_INFO: `/v3/projects/b30b71f082204338aba8410cdf87a5fc/users/2d460337950b45d3825b38c627f2dcaa/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.927687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30b71f082204338aba8410cdf87a5fc, user_id=2d460337950b45d3825b38c627f2dcaa, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:33.930342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:33.940182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1321/2635] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:33 2022] PUT /identity/v3/projects/b30b71f082204338aba8410cdf87a5fc/users/2d460337950b45d3825b38c627f2dcaa/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:33.951307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad65932-3d84-446a-accb-c1a4ea00486b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:33.951592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad65932-3d84-446a-accb-c1a4ea00486b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:33.951862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad65932-3d84-446a-accb-c1a4ea00486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:33.979961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4ad65932-3d84-446a-accb-c1a4ea00486b None None] MFA Rules not processed for user `2d460337950b45d3825b38c627f2dcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:33.981209 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4ad65932-3d84-446a-accb-c1a4ea00486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:34.045910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1315/2636] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:34.056766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:34.058498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:34.115658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28b6b1-b1fc-433b-88af-d74b7f2e4fbe admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:34.119269 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464ea6e-a58d-4077-aadf-e9af71da45a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:34.119868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464ea6e-a58d-4077-aadf-e9af71da45a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:34.120145 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464ea6e-a58d-4077-aadf-e9af71da45a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:34.120412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464ea6e-a58d-4077-aadf-e9af71da45a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:34.129169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464ea6e-a58d-4077-aadf-e9af71da45a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:34.130579 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464ea6e-a58d-4077-aadf-e9af71da45a5 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:34.140289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1322/2637] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:34.675531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad65932-3d84-446a-accb-c1a4ea00486b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:34.736345 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad65932-3d84-446a-accb-c1a4ea00486b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:34.740803 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:34.741429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:34.741712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:34.741985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] PATH_INFO: `/v3/users/94ec95f6f4b642abaf6add2ada5fafc5` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:34.743132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ec95f6f4b642abaf6add2ada5fafc5)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:34.745122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:34.790426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ec95f6f4b642abaf6add2ada5fafc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:34.792522 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:34.798536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ec95f6f4b642abaf6add2ada5fafc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:34.799569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ec95f6f4b642abaf6add2ada5fafc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:34.800206 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ec95f6f4b642abaf6add2ada5fafc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:34.807526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ec95f6f4b642abaf6add2ada5fafc5'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:34.825113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1316/2638] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:34 2022] DELETE /identity/v3/users/94ec95f6f4b642abaf6add2ada5fafc5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:34.829674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179358-d675-4031-a36c-c6c8c6862805 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:34.829956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179358-d675-4031-a36c-c6c8c6862805 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:34.830225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179358-d675-4031-a36c-c6c8c6862805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:34.849400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-31179358-d675-4031-a36c-c6c8c6862805 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:34.850588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-31179358-d675-4031-a36c-c6c8c6862805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:34.906867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1323/2639] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:34.916578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:34.917952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:34.974898 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec6e60-0ac7-4d1f-84d3-4cf4d4537b42 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:34.978560 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce308b-14c9-4884-8ec3-516fa91722bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:34.979190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce308b-14c9-4884-8ec3-516fa91722bb service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:34.979475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce308b-14c9-4884-8ec3-516fa91722bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:34.979747 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce308b-14c9-4884-8ec3-516fa91722bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:34.988484 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce308b-14c9-4884-8ec3-516fa91722bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:34.989901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce308b-14c9-4884-8ec3-516fa91722bb service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:34.999314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1317/2640] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:35.261324 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31179358-d675-4031-a36c-c6c8c6862805 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:35.262770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-31179358-d675-4031-a36c-c6c8c6862805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:35.276735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31179358-d675-4031-a36c-c6c8c6862805 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:35.281535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:35.282546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:35.282546 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:35.282803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] PATH_INFO: `/v3/projects/b92feee8f1ee4073abfae09015da1705` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:35.283860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b92feee8f1ee4073abfae09015da1705)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:35.285760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:35.332410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b92feee8f1ee4073abfae09015da1705', 'request_id': 'req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.340610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b92feee8f1ee4073abfae09015da1705', 'request_id': 'req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.347108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b92feee8f1ee4073abfae09015da1705', 'request_id': 'req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.357212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] The token cache is being invalidate because project b92feee8f1ee4073abfae09015da1705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:35.357495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.358115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.359601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1324/2641] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:35 2022] DELETE /identity/v3/projects/b92feee8f1ee4073abfae09015da1705 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:35.363643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce308b-14c9-4884-8ec3-516fa91722bb service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:35.373656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce308b-14c9-4884-8ec3-516fa91722bb service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:35.429583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:35.430741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:35.431059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:35.431327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] PATH_INFO: `/v3/users/2d460337950b45d3825b38c627f2dcaa` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:35.432439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d460337950b45d3825b38c627f2dcaa)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:35.434327 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:35.477357 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d460337950b45d3825b38c627f2dcaa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.484084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d460337950b45d3825b38c627f2dcaa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.484724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d460337950b45d3825b38c627f2dcaa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.485320 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d460337950b45d3825b38c627f2dcaa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.492130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0bbb1a78-cfd7-4fea-b267-2ca13546be36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d460337950b45d3825b38c627f2dcaa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.499710 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:35.512298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1318/2642] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:35 2022] DELETE /identity/v3/users/2d460337950b45d3825b38c627f2dcaa => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:35.786283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:35.791286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:35.807533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ead2f-8fd5-4cf8-a8bc-69931012adbb admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:35.821099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:35.868990 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:35.869533 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:35.869809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:35.870076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] PATH_INFO: `/v3/projects/b30b71f082204338aba8410cdf87a5fc` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:35.871201 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30b71f082204338aba8410cdf87a5fc)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:35.873093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:35.918016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30b71f082204338aba8410cdf87a5fc', 'request_id': 'req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.925617 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30b71f082204338aba8410cdf87a5fc', 'request_id': 'req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.932558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30b71f082204338aba8410cdf87a5fc', 'request_id': 'req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.944420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] The token cache is being invalidate because project b30b71f082204338aba8410cdf87a5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:35.944701 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.945487 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-201711fb-ed70-4e7b-bf69-100ebb1e0bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:35.947163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1325/2643] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:35 2022] DELETE /identity/v3/projects/b30b71f082204338aba8410cdf87a5fc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:35.951981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67694118-afb6-4e0a-b491-211e262d4642 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:35.952464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67694118-afb6-4e0a-b491-211e262d4642 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:35.952578 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67694118-afb6-4e0a-b491-211e262d4642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:35.972192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-67694118-afb6-4e0a-b491-211e262d4642 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:35.973404 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-67694118-afb6-4e0a-b491-211e262d4642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:36.030468 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1319/2644] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:36.038039 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb51ef-41b1-472b-80fd-c7ae97b9721e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.038327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb51ef-41b1-472b-80fd-c7ae97b9721e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.038599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb51ef-41b1-472b-80fd-c7ae97b9721e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.058695 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-dffb51ef-41b1-472b-80fd-c7ae97b9721e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:36.059919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-dffb51ef-41b1-472b-80fd-c7ae97b9721e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:36.067703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1326/2645] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:36.071789 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67694118-afb6-4e0a-b491-211e262d4642 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.081226 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67694118-afb6-4e0a-b491-211e262d4642 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.085795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.086909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.087203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.087475 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.088124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:36.088425 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.089991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.101269 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1320/2646] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:06:36.106034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb51ef-41b1-472b-80fd-c7ae97b9721e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.114963 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb51ef-41b1-472b-80fd-c7ae97b9721e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.119698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3e7a5-b496-4bdf-8778-8692524d44f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.120227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e7a5-b496-4bdf-8778-8692524d44f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.120505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e7a5-b496-4bdf-8778-8692524d44f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.120772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e7a5-b496-4bdf-8778-8692524d44f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.121389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3e7a5-b496-4bdf-8778-8692524d44f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.123302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3e7a5-b496-4bdf-8778-8692524d44f0 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.150651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1327/2647] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:36 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:36.154702 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.168147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac66eee-43f7-45c0-bb7a-f966f4054d39 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.223509 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fe1ec-2fc7-430e-8643-7b620bce8d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.224112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fe1ec-2fc7-430e-8643-7b620bce8d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.224394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fe1ec-2fc7-430e-8643-7b620bce8d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.224665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fe1ec-2fc7-430e-8643-7b620bce8d18 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.225313 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fe1ec-2fc7-430e-8643-7b620bce8d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.227846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fe1ec-2fc7-430e-8643-7b620bce8d18 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.248197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1321/2648] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:36 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:36.252437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3e7a5-b496-4bdf-8778-8692524d44f0 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.262490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3e7a5-b496-4bdf-8778-8692524d44f0 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.267008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8961e-a45e-4710-8730-354ef400b34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.267593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8961e-a45e-4710-8730-354ef400b34c admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.267868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8961e-a45e-4710-8730-354ef400b34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.268374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8961e-a45e-4710-8730-354ef400b34c admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.269007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8961e-a45e-4710-8730-354ef400b34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.270603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8961e-a45e-4710-8730-354ef400b34c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.280700 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1328/2649] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:36.284385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fe1ec-2fc7-430e-8643-7b620bce8d18 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.294620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fe1ec-2fc7-430e-8643-7b620bce8d18 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.299494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dd57e-adbb-4f10-a695-c5783073fd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.300079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd57e-adbb-4f10-a695-c5783073fd34 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.300363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd57e-adbb-4f10-a695-c5783073fd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.300639 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd57e-adbb-4f10-a695-c5783073fd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.301272 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128dd57e-adbb-4f10-a695-c5783073fd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.302835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128dd57e-adbb-4f10-a695-c5783073fd34 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.314106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1322/2650] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:36.317944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8961e-a45e-4710-8730-354ef400b34c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.328212 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8961e-a45e-4710-8730-354ef400b34c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.332694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cb95-0d4d-4e55-b746-261c3301b3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.333325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cb95-0d4d-4e55-b746-261c3301b3f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.333599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cb95-0d4d-4e55-b746-261c3301b3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.333868 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cb95-0d4d-4e55-b746-261c3301b3f2 admin admin] PATH_INFO: `/v3/projects/912704429a364461939492b33bc712ce/users/1817d51f07624f429f196463bc5ada7b/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.348600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03cb95-0d4d-4e55-b746-261c3301b3f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=912704429a364461939492b33bc712ce, user_id=1817d51f07624f429f196463bc5ada7b, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.351219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03cb95-0d4d-4e55-b746-261c3301b3f2 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.363690 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1329/2651] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:36 2022] PUT /identity/v3/projects/912704429a364461939492b33bc712ce/users/1817d51f07624f429f196463bc5ada7b/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:36.373948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1900748-f74c-46c3-8278-64550093b450 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.374176 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1900748-f74c-46c3-8278-64550093b450 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.374448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1900748-f74c-46c3-8278-64550093b450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.403923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b1900748-f74c-46c3-8278-64550093b450 None None] MFA Rules not processed for user `1817d51f07624f429f196463bc5ada7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:36.405143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b1900748-f74c-46c3-8278-64550093b450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:36.448797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:36.470586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1323/2652] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:36.475185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cb95-0d4d-4e55-b746-261c3301b3f2 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.518498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:36.537226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cb95-0d4d-4e55-b746-261c3301b3f2 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.541843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6d0e3-7abb-43aa-9dd4-e3c5752757cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.542439 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6d0e3-7abb-43aa-9dd4-e3c5752757cf admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.542718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6d0e3-7abb-43aa-9dd4-e3c5752757cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.543018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6d0e3-7abb-43aa-9dd4-e3c5752757cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.543650 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6d0e3-7abb-43aa-9dd4-e3c5752757cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.545538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6d0e3-7abb-43aa-9dd4-e3c5752757cf admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.572687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1330/2653] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:36 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:36.576935 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1900748-f74c-46c3-8278-64550093b450 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.588263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1900748-f74c-46c3-8278-64550093b450 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.641775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d48efd-6ba4-4bdc-b3e6-05bb76b21922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.642297 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d48efd-6ba4-4bdc-b3e6-05bb76b21922 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.642569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d48efd-6ba4-4bdc-b3e6-05bb76b21922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.642834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d48efd-6ba4-4bdc-b3e6-05bb76b21922 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.643469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d48efd-6ba4-4bdc-b3e6-05bb76b21922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.645343 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d48efd-6ba4-4bdc-b3e6-05bb76b21922 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.666114 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1324/2654] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:36 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:36.670216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6d0e3-7abb-43aa-9dd4-e3c5752757cf admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.681076 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6d0e3-7abb-43aa-9dd4-e3c5752757cf admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.685628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcc9bd-5e74-4b97-b351-a5a63df54a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.686213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcc9bd-5e74-4b97-b351-a5a63df54a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.686489 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcc9bd-5e74-4b97-b351-a5a63df54a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.686758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcc9bd-5e74-4b97-b351-a5a63df54a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.687419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcc9bd-5e74-4b97-b351-a5a63df54a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.688997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcc9bd-5e74-4b97-b351-a5a63df54a7b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.700181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1331/2655] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:36.704365 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d48efd-6ba4-4bdc-b3e6-05bb76b21922 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.714212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d48efd-6ba4-4bdc-b3e6-05bb76b21922 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.718948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d002f90-038e-496e-9cfb-385313ebeaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.719590 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002f90-038e-496e-9cfb-385313ebeaea admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.719867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002f90-038e-496e-9cfb-385313ebeaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.720162 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002f90-038e-496e-9cfb-385313ebeaea admin admin] PATH_INFO: `/v3/projects/2092fd2311a84965bebd7d6b88968dd7/users/2ab33865323d4c3e97c27ac2eaf4efda/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.734783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d002f90-038e-496e-9cfb-385313ebeaea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2092fd2311a84965bebd7d6b88968dd7, user_id=2ab33865323d4c3e97c27ac2eaf4efda, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.737397 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d002f90-038e-496e-9cfb-385313ebeaea admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.750391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1325/2656] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:36 2022] PUT /identity/v3/projects/2092fd2311a84965bebd7d6b88968dd7/users/2ab33865323d4c3e97c27ac2eaf4efda/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:36.754205 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcc9bd-5e74-4b97-b351-a5a63df54a7b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.764479 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcc9bd-5e74-4b97-b351-a5a63df54a7b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.820032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50513b-9eab-43c8-8029-7f01c264d94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.821261 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50513b-9eab-43c8-8029-7f01c264d94b admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.821567 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50513b-9eab-43c8-8029-7f01c264d94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.821839 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50513b-9eab-43c8-8029-7f01c264d94b admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.822472 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50513b-9eab-43c8-8029-7f01c264d94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.824090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50513b-9eab-43c8-8029-7f01c264d94b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.833973 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1332/2657] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:36.838130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d002f90-038e-496e-9cfb-385313ebeaea admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:36.848187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d002f90-038e-496e-9cfb-385313ebeaea admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:36.852743 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:36.853383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.853660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.853927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] PATH_INFO: `/v3/projects/2092fd2311a84965bebd7d6b88968dd7/users/2ab33865323d4c3e97c27ac2eaf4efda/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.855863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2092fd2311a84965bebd7d6b88968dd7, user_id=2ab33865323d4c3e97c27ac2eaf4efda, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:36.858448 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:36.868934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1326/2658] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:36 2022] PUT /identity/v3/projects/2092fd2311a84965bebd7d6b88968dd7/users/2ab33865323d4c3e97c27ac2eaf4efda/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:36.934058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100821-a734-4ed2-ad2d-36e0725a2ce6 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:36.934277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100821-a734-4ed2-ad2d-36e0725a2ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:36.934514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d100821-a734-4ed2-ad2d-36e0725a2ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:36.962352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6d100821-a734-4ed2-ad2d-36e0725a2ce6 None None] MFA Rules not processed for user `2ab33865323d4c3e97c27ac2eaf4efda`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:36.963597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6d100821-a734-4ed2-ad2d-36e0725a2ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:37.028678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1333/2659] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:37.040154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:37.041991 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:37.099640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bfd5c-02f7-4de8-bad6-d9567d40e7e4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:37.103781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8a5ea-54fa-4958-83c5-44926c299fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:37.104369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8a5ea-54fa-4958-83c5-44926c299fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:37.104643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8a5ea-54fa-4958-83c5-44926c299fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:37.104910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8a5ea-54fa-4958-83c5-44926c299fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:37.113818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8a5ea-54fa-4958-83c5-44926c299fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:37.115310 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8a5ea-54fa-4958-83c5-44926c299fd6 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:37.125305 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1327/2660] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:37.199513 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100821-a734-4ed2-ad2d-36e0725a2ce6 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:37.200939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6d100821-a734-4ed2-ad2d-36e0725a2ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:37.213914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d100821-a734-4ed2-ad2d-36e0725a2ce6 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:37.217500 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b3dd3-cc49-4d2a-b595-c6e4d569cfb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:37.218085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b3dd3-cc49-4d2a-b595-c6e4d569cfb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:37.218370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b3dd3-cc49-4d2a-b595-c6e4d569cfb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:37.218747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b3dd3-cc49-4d2a-b595-c6e4d569cfb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:37.251612 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:37.274312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b3dd3-cc49-4d2a-b595-c6e4d569cfb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:37.275753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b3dd3-cc49-4d2a-b595-c6e4d569cfb5 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:37.284440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1334/2661] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:37.540422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8a5ea-54fa-4958-83c5-44926c299fd6 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:37.582447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:37.602600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8a5ea-54fa-4958-83c5-44926c299fd6 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:37.607108 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:37.607728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:37.608005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:37.608274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] PATH_INFO: `/v3/users/1817d51f07624f429f196463bc5ada7b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:37.609378 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1817d51f07624f429f196463bc5ada7b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:37.611321 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:37.656230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1817d51f07624f429f196463bc5ada7b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:37.663309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1817d51f07624f429f196463bc5ada7b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:37.663982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1817d51f07624f429f196463bc5ada7b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:37.664699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1817d51f07624f429f196463bc5ada7b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:37.671623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1817d51f07624f429f196463bc5ada7b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:37.689073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1328/2662] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:37 2022] DELETE /identity/v3/users/1817d51f07624f429f196463bc5ada7b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:37.693683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9595e-82cb-4068-a849-affd5ca3c434 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:37.693965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9595e-82cb-4068-a849-affd5ca3c434 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:37.694231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9595e-82cb-4068-a849-affd5ca3c434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:37.713369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-31b9595e-82cb-4068-a849-affd5ca3c434 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:37.714483 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-31b9595e-82cb-4068-a849-affd5ca3c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:37.771507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1335/2663] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:37.781740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:37.783129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:37.840672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c8fec-a03f-4674-9a71-a98042ae5b06 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:37.844585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2b4be-afe5-4b9c-93cd-a74dff7d39ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:37.845189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2b4be-afe5-4b9c-93cd-a74dff7d39ff service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:37.845467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2b4be-afe5-4b9c-93cd-a74dff7d39ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:37.845736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2b4be-afe5-4b9c-93cd-a74dff7d39ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:37.855135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2b4be-afe5-4b9c-93cd-a74dff7d39ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:37.856556 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2b4be-afe5-4b9c-93cd-a74dff7d39ff service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:37.866026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1329/2664] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:38.142115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9595e-82cb-4068-a849-affd5ca3c434 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:38.143582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-31b9595e-82cb-4068-a849-affd5ca3c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:38.156549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9595e-82cb-4068-a849-affd5ca3c434 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:38.161091 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:38.161714 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:38.161992 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:38.162263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] PATH_INFO: `/v3/projects/912704429a364461939492b33bc712ce` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:38.163406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=912704429a364461939492b33bc712ce)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:38.165291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:38.214869 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '912704429a364461939492b33bc712ce', 'request_id': 'req-6f8e086a-eac1-45f4-94fa-4bbb155a8604'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.223271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '912704429a364461939492b33bc712ce', 'request_id': 'req-6f8e086a-eac1-45f4-94fa-4bbb155a8604'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.230065 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '912704429a364461939492b33bc712ce', 'request_id': 'req-6f8e086a-eac1-45f4-94fa-4bbb155a8604'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.240784 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] The token cache is being invalidate because project 912704429a364461939492b33bc712ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:38.241070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.241723 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.243761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1336/2665] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:38 2022] DELETE /identity/v3/projects/912704429a364461939492b33bc712ce => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:38.247893 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2b4be-afe5-4b9c-93cd-a74dff7d39ff service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:38.259853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2b4be-afe5-4b9c-93cd-a74dff7d39ff service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:38.322463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:38.323140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:38.323419 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:38.323731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] PATH_INFO: `/v3/users/2ab33865323d4c3e97c27ac2eaf4efda` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:38.324897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab33865323d4c3e97c27ac2eaf4efda)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:38.326916 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:38.329659 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:38.377188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab33865323d4c3e97c27ac2eaf4efda'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.385331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab33865323d4c3e97c27ac2eaf4efda'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.386082 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab33865323d4c3e97c27ac2eaf4efda'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.387227 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab33865323d4c3e97c27ac2eaf4efda'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.396128 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-acfa6144-ea05-46d0-adfa-ed5aa5d7ad48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab33865323d4c3e97c27ac2eaf4efda'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.420533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1330/2666] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:38 2022] DELETE /identity/v3/users/2ab33865323d4c3e97c27ac2eaf4efda => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:38.690041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:38.691530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:38.704343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e086a-eac1-45f4-94fa-4bbb155a8604 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:38.760315 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:38.760856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:38.761133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:38.761418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] PATH_INFO: `/v3/projects/2092fd2311a84965bebd7d6b88968dd7` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:38.762507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2092fd2311a84965bebd7d6b88968dd7)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:38.764436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:38.765851 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:38.811614 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2092fd2311a84965bebd7d6b88968dd7', 'request_id': 'req-79afbc53-f722-45af-a6f9-c7bb15fea97d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.818813 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2092fd2311a84965bebd7d6b88968dd7', 'request_id': 'req-79afbc53-f722-45af-a6f9-c7bb15fea97d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.825302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2092fd2311a84965bebd7d6b88968dd7', 'request_id': 'req-79afbc53-f722-45af-a6f9-c7bb15fea97d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.835469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] The token cache is being invalidate because project 2092fd2311a84965bebd7d6b88968dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:38.835755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.836378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-79afbc53-f722-45af-a6f9-c7bb15fea97d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:38.837833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1337/2667] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:38 2022] DELETE /identity/v3/projects/2092fd2311a84965bebd7d6b88968dd7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:38.842778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3eb67-6659-4b27-9b30-d38cfa642496 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:38.843125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3eb67-6659-4b27-9b30-d38cfa642496 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:38.843406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3eb67-6659-4b27-9b30-d38cfa642496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:38.865268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0db3eb67-6659-4b27-9b30-d38cfa642496 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:38.866506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0db3eb67-6659-4b27-9b30-d38cfa642496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:38.926655 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1331/2668] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:38.934946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9876d-e254-4edf-bc98-caea92303a5e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:38.935236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9876d-e254-4edf-bc98-caea92303a5e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:38.935505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9876d-e254-4edf-bc98-caea92303a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:38.956935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a0a9876d-e254-4edf-bc98-caea92303a5e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:38.958135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a0a9876d-e254-4edf-bc98-caea92303a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:38.967115 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1338/2669] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:38.971322 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3eb67-6659-4b27-9b30-d38cfa642496 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:38.981494 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3eb67-6659-4b27-9b30-d38cfa642496 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:38.986047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:38.986641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:38.986945 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:38.987217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:38.987904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:38.988198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:38.989751 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:39.001181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1332/2670] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:06:39.005553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9876d-e254-4edf-bc98-caea92303a5e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:39.015597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9876d-e254-4edf-bc98-caea92303a5e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:39.020337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0ac43-64fa-4239-b5d0-8ffd7c416042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:39.020943 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ac43-64fa-4239-b5d0-8ffd7c416042 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:39.021221 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ac43-64fa-4239-b5d0-8ffd7c416042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:39.021495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ac43-64fa-4239-b5d0-8ffd7c416042 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:39.022125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0ac43-64fa-4239-b5d0-8ffd7c416042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:39.025410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0ac43-64fa-4239-b5d0-8ffd7c416042 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:39.054912 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1339/2671] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:39 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:39.059239 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:39.072164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5db823-e35d-4041-8ade-2625fcf816d5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:39.146438 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd653d70-b2dc-4a95-b858-e837c203d0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:39.147079 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd653d70-b2dc-4a95-b858-e837c203d0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:39.147363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd653d70-b2dc-4a95-b858-e837c203d0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:39.147642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd653d70-b2dc-4a95-b858-e837c203d0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:39.148265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd653d70-b2dc-4a95-b858-e837c203d0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:39.151022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd653d70-b2dc-4a95-b858-e837c203d0f3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:39.167309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:39.174421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1333/2672] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:39 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:39.178118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0ac43-64fa-4239-b5d0-8ffd7c416042 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:39.188172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0ac43-64fa-4239-b5d0-8ffd7c416042 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:39.192666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43a06b-8347-4578-ba9c-11a7d7bcc40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:39.193247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43a06b-8347-4578-ba9c-11a7d7bcc40c admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:39.193521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43a06b-8347-4578-ba9c-11a7d7bcc40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:39.193785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43a06b-8347-4578-ba9c-11a7d7bcc40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:39.194421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43a06b-8347-4578-ba9c-11a7d7bcc40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:39.196017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43a06b-8347-4578-ba9c-11a7d7bcc40c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:39.205531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1340/2673] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:39.209579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd653d70-b2dc-4a95-b858-e837c203d0f3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:39.220412 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd653d70-b2dc-4a95-b858-e837c203d0f3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:39.225445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c21865-d944-48ff-8713-b984a5dd3238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:39.226034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21865-d944-48ff-8713-b984a5dd3238 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:39.226308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21865-d944-48ff-8713-b984a5dd3238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:39.226599 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21865-d944-48ff-8713-b984a5dd3238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:39.227839 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c21865-d944-48ff-8713-b984a5dd3238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:39.229389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c21865-d944-48ff-8713-b984a5dd3238 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:39.239902 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1334/2674] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:39.244148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43a06b-8347-4578-ba9c-11a7d7bcc40c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:39.254001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43a06b-8347-4578-ba9c-11a7d7bcc40c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:39.258585 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:39.259280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:39.259580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:39.259826 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] PATH_INFO: `/v3/projects/6f5b27dc9eb342beae98d07663851170/users/1a30312c89aa4661ada14849e4f1cbac/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:39.274598 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5b27dc9eb342beae98d07663851170, user_id=1a30312c89aa4661ada14849e4f1cbac, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:39.277213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:39.290935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1341/2675] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:39 2022] PUT /identity/v3/projects/6f5b27dc9eb342beae98d07663851170/users/1a30312c89aa4661ada14849e4f1cbac/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:39.302849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48429442-e40c-4d03-945c-cdae05a6bd19 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:39.303385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48429442-e40c-4d03-945c-cdae05a6bd19 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:39.303485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48429442-e40c-4d03-945c-cdae05a6bd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:39.330818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-48429442-e40c-4d03-945c-cdae05a6bd19 None None] MFA Rules not processed for user `1a30312c89aa4661ada14849e4f1cbac`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:39.332037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-48429442-e40c-4d03-945c-cdae05a6bd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:39.398194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1335/2676] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:39.410935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:39.412638 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:39.471952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fabe0-cbff-48b8-9e84-36ae0897bb00 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:39.476194 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d03655-8b14-425f-851e-1ba39c69788e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:39.477300 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03655-8b14-425f-851e-1ba39c69788e service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:39.477584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03655-8b14-425f-851e-1ba39c69788e service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:39.477852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03655-8b14-425f-851e-1ba39c69788e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:39.486631 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d03655-8b14-425f-851e-1ba39c69788e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:39.488077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d03655-8b14-425f-851e-1ba39c69788e service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:39.497601 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1342/2677] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:46.251773 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48429442-e40c-4d03-945c-cdae05a6bd19 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:46.313532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48429442-e40c-4d03-945c-cdae05a6bd19 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:46.317944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00493292-4955-4281-9912-6980871d0546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:46.318557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00493292-4955-4281-9912-6980871d0546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:46.318838 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00493292-4955-4281-9912-6980871d0546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:46.319132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00493292-4955-4281-9912-6980871d0546 admin admin] PATH_INFO: `/v3/users/1a30312c89aa4661ada14849e4f1cbac` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:46.320216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00493292-4955-4281-9912-6980871d0546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a30312c89aa4661ada14849e4f1cbac)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:46.322653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00493292-4955-4281-9912-6980871d0546 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:46.369130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00493292-4955-4281-9912-6980871d0546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a30312c89aa4661ada14849e4f1cbac'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.375927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00493292-4955-4281-9912-6980871d0546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a30312c89aa4661ada14849e4f1cbac'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.376569 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00493292-4955-4281-9912-6980871d0546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a30312c89aa4661ada14849e4f1cbac'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.377166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00493292-4955-4281-9912-6980871d0546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a30312c89aa4661ada14849e4f1cbac'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.384833 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-00493292-4955-4281-9912-6980871d0546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a30312c89aa4661ada14849e4f1cbac'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.396959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:46.405396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1336/2678] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:46 2022] DELETE /identity/v3/users/1a30312c89aa4661ada14849e4f1cbac => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:46.410020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c82ac9-70e9-4f50-80fb-f49f66141d88 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:46.410437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c82ac9-70e9-4f50-80fb-f49f66141d88 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:46.410574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c82ac9-70e9-4f50-80fb-f49f66141d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:46.429286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-42c82ac9-70e9-4f50-80fb-f49f66141d88 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:46.431036 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-42c82ac9-70e9-4f50-80fb-f49f66141d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:46.454460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:46.494732 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1343/2679] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:46.504437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00493292-4955-4281-9912-6980871d0546 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:46.505862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-00493292-4955-4281-9912-6980871d0546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:46.567467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00493292-4955-4281-9912-6980871d0546 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:46.573309 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953e789-c4bd-4ed7-a88f-17feea79f786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:46.573910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e789-c4bd-4ed7-a88f-17feea79f786 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:46.574217 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e789-c4bd-4ed7-a88f-17feea79f786 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:46.574512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e789-c4bd-4ed7-a88f-17feea79f786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:46.584795 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953e789-c4bd-4ed7-a88f-17feea79f786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:46.586282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953e789-c4bd-4ed7-a88f-17feea79f786 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:46.596301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1337/2680] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:46.877409 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c82ac9-70e9-4f50-80fb-f49f66141d88 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:46.878821 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-42c82ac9-70e9-4f50-80fb-f49f66141d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:46.891486 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c82ac9-70e9-4f50-80fb-f49f66141d88 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:46.897101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:46.897838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:46.898135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:46.898389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] PATH_INFO: `/v3/projects/6f5b27dc9eb342beae98d07663851170` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:46.900148 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5b27dc9eb342beae98d07663851170)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:46.902099 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:46.948471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5b27dc9eb342beae98d07663851170', 'request_id': 'req-50092175-a59f-4cda-a892-d5bdeb1a1678'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.955335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5b27dc9eb342beae98d07663851170', 'request_id': 'req-50092175-a59f-4cda-a892-d5bdeb1a1678'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.962495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5b27dc9eb342beae98d07663851170', 'request_id': 'req-50092175-a59f-4cda-a892-d5bdeb1a1678'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.972724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] The token cache is being invalidate because project 6f5b27dc9eb342beae98d07663851170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:46.973007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.973629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-50092175-a59f-4cda-a892-d5bdeb1a1678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:46.975140 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1344/2681] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:46 2022] DELETE /identity/v3/projects/6f5b27dc9eb342beae98d07663851170 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:46.981530 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16da5b-777b-48ba-afa8-3db99b1742b0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:46.981820 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16da5b-777b-48ba-afa8-3db99b1742b0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:46.982092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16da5b-777b-48ba-afa8-3db99b1742b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.000909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8c16da5b-777b-48ba-afa8-3db99b1742b0 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:47.002095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8c16da5b-777b-48ba-afa8-3db99b1742b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:47.059073 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1338/2682] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:47.066710 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d498110-2ee9-415e-80b3-d2d0dca8bbbf None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.067130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d498110-2ee9-415e-80b3-d2d0dca8bbbf None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.067429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d498110-2ee9-415e-80b3-d2d0dca8bbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.087618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9d498110-2ee9-415e-80b3-d2d0dca8bbbf None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:47.088814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9d498110-2ee9-415e-80b3-d2d0dca8bbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:47.096739 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1345/2683] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:47.100976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16da5b-777b-48ba-afa8-3db99b1742b0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:47.110857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16da5b-777b-48ba-afa8-3db99b1742b0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:47.115414 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:47.116010 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.116294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.116576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.117216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:47.117518 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:47.119119 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:47.130842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1339/2684] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:06:47.135574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d498110-2ee9-415e-80b3-d2d0dca8bbbf None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:47.144625 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d498110-2ee9-415e-80b3-d2d0dca8bbbf None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:47.149757 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bb8c-a295-42bd-9337-4166fca68684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:47.150341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bb8c-a295-42bd-9337-4166fca68684 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.150618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bb8c-a295-42bd-9337-4166fca68684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.150902 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bb8c-a295-42bd-9337-4166fca68684 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.151540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3bb8c-a295-42bd-9337-4166fca68684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:47.153421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3bb8c-a295-42bd-9337-4166fca68684 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:47.181014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1346/2685] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:47 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:47.185184 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:47.194860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb270f-ff42-4e70-874e-bcadd8464ef2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:47.250531 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180c512-fda6-4fe7-8c7a-2b3aaaa868a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:47.251078 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180c512-fda6-4fe7-8c7a-2b3aaaa868a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.251361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180c512-fda6-4fe7-8c7a-2b3aaaa868a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.251653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180c512-fda6-4fe7-8c7a-2b3aaaa868a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.252249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180c512-fda6-4fe7-8c7a-2b3aaaa868a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:47.254147 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180c512-fda6-4fe7-8c7a-2b3aaaa868a0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:47.274598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1340/2686] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:47 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:47.278831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bb8c-a295-42bd-9337-4166fca68684 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:47.288793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bb8c-a295-42bd-9337-4166fca68684 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:47.293518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58f81f-99f3-4423-98ca-f7d613f3829b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:47.294118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f81f-99f3-4423-98ca-f7d613f3829b admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.294398 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f81f-99f3-4423-98ca-f7d613f3829b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.294670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58f81f-99f3-4423-98ca-f7d613f3829b admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.295365 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58f81f-99f3-4423-98ca-f7d613f3829b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:47.296965 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58f81f-99f3-4423-98ca-f7d613f3829b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:47.307467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1347/2687] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:47.311160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180c512-fda6-4fe7-8c7a-2b3aaaa868a0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:47.320804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180c512-fda6-4fe7-8c7a-2b3aaaa868a0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:47.326026 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9d578-7c9e-4120-b21a-df1d928d922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:47.326603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9d578-7c9e-4120-b21a-df1d928d922d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.326886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9d578-7c9e-4120-b21a-df1d928d922d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.327185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9d578-7c9e-4120-b21a-df1d928d922d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.327811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9d578-7c9e-4120-b21a-df1d928d922d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:47.329360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9d578-7c9e-4120-b21a-df1d928d922d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:47.340545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1341/2688] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:47.344461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58f81f-99f3-4423-98ca-f7d613f3829b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:47.354169 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58f81f-99f3-4423-98ca-f7d613f3829b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:47.358698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:47.359359 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.359640 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.359919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] PATH_INFO: `/v3/projects/52575749f6f6409b947c28bb16649eb7/users/fe67ba70aef540e98dc56fe609077f93/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.374934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] RBAC: Authorizing `identity:create_grant(project_id=52575749f6f6409b947c28bb16649eb7, user_id=fe67ba70aef540e98dc56fe609077f93, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:47.377581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:47.390512 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1348/2689] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:47 2022] PUT /identity/v3/projects/52575749f6f6409b947c28bb16649eb7/users/fe67ba70aef540e98dc56fe609077f93/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:47.395561 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2bf2b-1065-4970-8325-fb8e3832f418 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.395867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2bf2b-1065-4970-8325-fb8e3832f418 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.396123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2bf2b-1065-4970-8325-fb8e3832f418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.404084 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:47.416104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f5e2bf2b-1065-4970-8325-fb8e3832f418 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:47.417312 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f5e2bf2b-1065-4970-8325-fb8e3832f418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:47.475360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1342/2690] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:47.484798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:47.486516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:47.539142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:47.544045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30befd63-9840-4193-b976-e4edc40eacda admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:47.548806 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2b81d-8262-4300-8b5f-42cdfd2c203d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:47.549474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2b81d-8262-4300-8b5f-42cdfd2c203d service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:47.549754 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2b81d-8262-4300-8b5f-42cdfd2c203d service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:47.550057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2b81d-8262-4300-8b5f-42cdfd2c203d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:47.560013 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2b81d-8262-4300-8b5f-42cdfd2c203d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:47.561427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2b81d-8262-4300-8b5f-42cdfd2c203d service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:47.570760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1349/2691] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:51.540780 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbebe0-133d-4729-b32c-88035bc37e2c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:51.542121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbebe0-133d-4729-b32c-88035bc37e2c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:51.542496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbebe0-133d-4729-b32c-88035bc37e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:51.572617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-12dbebe0-133d-4729-b32c-88035bc37e2c None None] MFA Rules not processed for user `fe67ba70aef540e98dc56fe609077f93`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:51.573846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-12dbebe0-133d-4729-b32c-88035bc37e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:51.642821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1343/2692] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:51.647462 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2b81d-8262-4300-8b5f-42cdfd2c203d service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:51.657928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2b81d-8262-4300-8b5f-42cdfd2c203d service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:51.662571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7780c07-1d08-4296-93e9-35f96c3d963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:51.663214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7780c07-1d08-4296-93e9-35f96c3d963f admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:51.663490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7780c07-1d08-4296-93e9-35f96c3d963f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:51.663761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7780c07-1d08-4296-93e9-35f96c3d963f admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:51.664432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7780c07-1d08-4296-93e9-35f96c3d963f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:51.666330 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7780c07-1d08-4296-93e9-35f96c3d963f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:51.693406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1350/2693] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:51 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:51.698183 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbebe0-133d-4729-b32c-88035bc37e2c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:51.708805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbebe0-133d-4729-b32c-88035bc37e2c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:51.780760 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e4c8e-5946-4a81-8ed6-7f760aef6680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:51.781437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e4c8e-5946-4a81-8ed6-7f760aef6680 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:51.781719 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e4c8e-5946-4a81-8ed6-7f760aef6680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:51.782116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e4c8e-5946-4a81-8ed6-7f760aef6680 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:51.782754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e4c8e-5946-4a81-8ed6-7f760aef6680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:51.784692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e4c8e-5946-4a81-8ed6-7f760aef6680 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:51.810664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1344/2694] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:51 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:06:51.814649 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7780c07-1d08-4296-93e9-35f96c3d963f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:51.831707 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7780c07-1d08-4296-93e9-35f96c3d963f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:51.837699 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82776d3-26c3-4a9c-9edd-61a2e6499a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:51.838319 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82776d3-26c3-4a9c-9edd-61a2e6499a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:51.838603 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82776d3-26c3-4a9c-9edd-61a2e6499a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:51.838874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82776d3-26c3-4a9c-9edd-61a2e6499a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:51.839549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82776d3-26c3-4a9c-9edd-61a2e6499a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:51.841154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82776d3-26c3-4a9c-9edd-61a2e6499a10 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:51.853073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1351/2695] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:51.857334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e4c8e-5946-4a81-8ed6-7f760aef6680 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:51.867878 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e4c8e-5946-4a81-8ed6-7f760aef6680 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:51.872924 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb895eb3-f5ca-4120-b72b-d0c9942f7bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:51.873622 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb895eb3-f5ca-4120-b72b-d0c9942f7bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:51.873900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb895eb3-f5ca-4120-b72b-d0c9942f7bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:51.874167 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb895eb3-f5ca-4120-b72b-d0c9942f7bbe admin admin] PATH_INFO: `/v3/projects/efedebc0f9044fa9bed771719507bd1d/users/f55d7e6e42a14247bbe2ae926eecf8c9/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:51.894099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb895eb3-f5ca-4120-b72b-d0c9942f7bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=efedebc0f9044fa9bed771719507bd1d, user_id=f55d7e6e42a14247bbe2ae926eecf8c9, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:51.897037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb895eb3-f5ca-4120-b72b-d0c9942f7bbe admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:51.909401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:51.917141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1345/2696] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:51 2022] PUT /identity/v3/projects/efedebc0f9044fa9bed771719507bd1d/users/f55d7e6e42a14247bbe2ae926eecf8c9/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:51.921220 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82776d3-26c3-4a9c-9edd-61a2e6499a10 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:51.931809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82776d3-26c3-4a9c-9edd-61a2e6499a10 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:51.993577 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:51.994196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:51.994473 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:51.994782 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:51.995461 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:51.997058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:52.007803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1352/2697] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:52.011874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb895eb3-f5ca-4120-b72b-d0c9942f7bbe admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:52.022376 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb895eb3-f5ca-4120-b72b-d0c9942f7bbe admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:52.027761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b547c-e2ad-44a8-97e1-8e591368a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:52.028415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b547c-e2ad-44a8-97e1-8e591368a810 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:52.028692 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b547c-e2ad-44a8-97e1-8e591368a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:52.028961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b547c-e2ad-44a8-97e1-8e591368a810 admin admin] PATH_INFO: `/v3/projects/efedebc0f9044fa9bed771719507bd1d/users/f55d7e6e42a14247bbe2ae926eecf8c9/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:52.030868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b547c-e2ad-44a8-97e1-8e591368a810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efedebc0f9044fa9bed771719507bd1d, user_id=f55d7e6e42a14247bbe2ae926eecf8c9, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:52.033496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b547c-e2ad-44a8-97e1-8e591368a810 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:52.045044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1346/2698] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:52 2022] PUT /identity/v3/projects/efedebc0f9044fa9bed771719507bd1d/users/f55d7e6e42a14247bbe2ae926eecf8c9/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:52.564372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:52.566369 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:52.717043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65047833-e0ba-4222-8834-31aa606b686a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:52.804004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:52.804633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:52.804914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:52.805228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:52.808195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:52.854871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:52.909570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:52.912308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:52.924082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1353/2699] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:56.895667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211c69-023c-4b0a-b694-d94af35bb164 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:56.896502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211c69-023c-4b0a-b694-d94af35bb164 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:56.896948 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211c69-023c-4b0a-b694-d94af35bb164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:56.929383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-df211c69-023c-4b0a-b694-d94af35bb164 None None] MFA Rules not processed for user `f55d7e6e42a14247bbe2ae926eecf8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:56.931164 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df211c69-023c-4b0a-b694-d94af35bb164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:57.017951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1347/2700] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:57.026871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:57.028492 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:57.107038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6432b-b38a-44c9-bcad-25664c5b54ce service cinder] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:57.113378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a81a5-665c-468f-8ced-e5b65d9bf545 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:57.113997 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a81a5-665c-468f-8ced-e5b65d9bf545 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:57.114280 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a81a5-665c-468f-8ced-e5b65d9bf545 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:57.114557 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a81a5-665c-468f-8ced-e5b65d9bf545 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:57.191258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a81a5-665c-468f-8ced-e5b65d9bf545 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:57.193130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a81a5-665c-468f-8ced-e5b65d9bf545 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:57.206108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1354/2701] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:57.407711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211c69-023c-4b0a-b694-d94af35bb164 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:57.409266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df211c69-023c-4b0a-b694-d94af35bb164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:57.546592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211c69-023c-4b0a-b694-d94af35bb164 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:57.556236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:57.557856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:57.558445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:57.558876 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] PATH_INFO: `/v3/users/0d4dbd7cc9c7411ab13ba708b28337dd` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:57.562157 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4dbd7cc9c7411ab13ba708b28337dd)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:57.564364 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:57.661254 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4dbd7cc9c7411ab13ba708b28337dd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:57.670439 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4dbd7cc9c7411ab13ba708b28337dd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:57.671288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4dbd7cc9c7411ab13ba708b28337dd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:57.672286 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4dbd7cc9c7411ab13ba708b28337dd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:57.684317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4dbd7cc9c7411ab13ba708b28337dd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:57.701172 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:57.716911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1348/2702] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:57 2022] DELETE /identity/v3/users/0d4dbd7cc9c7411ab13ba708b28337dd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:57.721635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f38b2-ad36-44ea-ac36-3c9220ee7d12 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:57.722059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f38b2-ad36-44ea-ac36-3c9220ee7d12 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:57.722656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f38b2-ad36-44ea-ac36-3c9220ee7d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:57.754442 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-ef6f38b2-ad36-44ea-ac36-3c9220ee7d12 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:57.755913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ef6f38b2-ad36-44ea-ac36-3c9220ee7d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:57.877006 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1355/2703] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:57.894054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:57.895797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:58.023797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391f53b-9f27-4cdc-8753-1d6a1495bc73 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:58.028886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93463d93-f9af-4814-bd44-afc60ff379d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:58.029593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93463d93-f9af-4814-bd44-afc60ff379d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:58.029953 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93463d93-f9af-4814-bd44-afc60ff379d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:58.030293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93463d93-f9af-4814-bd44-afc60ff379d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:58.054529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93463d93-f9af-4814-bd44-afc60ff379d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:58.054529 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93463d93-f9af-4814-bd44-afc60ff379d8 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:58.073150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1349/2704] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:58.526089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f38b2-ad36-44ea-ac36-3c9220ee7d12 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:58.527623 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ef6f38b2-ad36-44ea-ac36-3c9220ee7d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:58.535494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=100 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:58.545179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f38b2-ad36-44ea-ac36-3c9220ee7d12 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:58.552248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:58.553653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:58.554179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:58.554722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] PATH_INFO: `/v3/projects/10ef065e40774935b7ee06648edf5f46` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:58.558279 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ef065e40774935b7ee06648edf5f46)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:58.565343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:58.626173 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ef065e40774935b7ee06648edf5f46', 'request_id': 'req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.634810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ef065e40774935b7ee06648edf5f46', 'request_id': 'req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.641787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ef065e40774935b7ee06648edf5f46', 'request_id': 'req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.665038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] The token cache is being invalidate because project 10ef065e40774935b7ee06648edf5f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:58.665534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.666926 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68e0f7ac-3dd5-4cdb-b09b-abca6e50ca77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.668552 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1356/2705] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:58 2022] DELETE /identity/v3/projects/10ef065e40774935b7ee06648edf5f46 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:58.675583 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93463d93-f9af-4814-bd44-afc60ff379d8 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:58.689381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93463d93-f9af-4814-bd44-afc60ff379d8 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:58.758699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:58.759422 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:58.759706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:58.759981 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] PATH_INFO: `/v3/users/4cc108b2108f4670920af96beb2cc753` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:58.761143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc108b2108f4670920af96beb2cc753)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:58.763090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:58.819330 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc108b2108f4670920af96beb2cc753'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.832841 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc108b2108f4670920af96beb2cc753'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.833754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc108b2108f4670920af96beb2cc753'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.834304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc108b2108f4670920af96beb2cc753'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.845302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa96ec-ed49-4440-a445-e5fe3962752c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:58.845599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa96ec-ed49-4440-a445-e5fe3962752c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:58.845873 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa96ec-ed49-4440-a445-e5fe3962752c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:58.846160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc108b2108f4670920af96beb2cc753'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:58.865749 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-18aa96ec-ed49-4440-a445-e5fe3962752c None None] MFA Rules not processed for user `fe67ba70aef540e98dc56fe609077f93`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:58.867014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-18aa96ec-ed49-4440-a445-e5fe3962752c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:58.868539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1350/2706] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:06:58 2022] DELETE /identity/v3/users/4cc108b2108f4670920af96beb2cc753 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:58.932600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1357/2707] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:06:59.211822 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:59.213753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:59.240690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:59.332344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034156c-1972-4a6b-b2b2-18c8a7d01a2a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:59.338705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:59.340580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.340865 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.341136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] PATH_INFO: `/v3/projects/25f7e016c3f644dc9fb4d04ee290c397` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.342834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f7e016c3f644dc9fb4d04ee290c397)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:59.345342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:59.412846 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f7e016c3f644dc9fb4d04ee290c397', 'request_id': 'req-355acf15-d731-4661-8603-87a2e96acc5b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:59.424253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f7e016c3f644dc9fb4d04ee290c397', 'request_id': 'req-355acf15-d731-4661-8603-87a2e96acc5b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:59.438972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f7e016c3f644dc9fb4d04ee290c397', 'request_id': 'req-355acf15-d731-4661-8603-87a2e96acc5b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:59.451651 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] The token cache is being invalidate because project 25f7e016c3f644dc9fb4d04ee290c397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:06:59.452018 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:59.455153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-355acf15-d731-4661-8603-87a2e96acc5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:06:59.458976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1351/2708] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:06:59 2022] DELETE /identity/v3/projects/25f7e016c3f644dc9fb4d04ee290c397 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:06:59.462187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e27e7-cd67-4981-96e3-c3fe8829960f None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.462495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e27e7-cd67-4981-96e3-c3fe8829960f None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.462793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e27e7-cd67-4981-96e3-c3fe8829960f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.488681 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-1e4e27e7-cd67-4981-96e3-c3fe8829960f None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:59.490082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1e4e27e7-cd67-4981-96e3-c3fe8829960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:59.567153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1358/2709] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:59.575021 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56aca8-108a-42ee-bf62-4d2f0156aefa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.575504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56aca8-108a-42ee-bf62-4d2f0156aefa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.575786 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56aca8-108a-42ee-bf62-4d2f0156aefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.595467 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5c56aca8-108a-42ee-bf62-4d2f0156aefa None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:59.596667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5c56aca8-108a-42ee-bf62-4d2f0156aefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:59.605037 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1352/2710] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:59.609380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e27e7-cd67-4981-96e3-c3fe8829960f None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:59.619913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e27e7-cd67-4981-96e3-c3fe8829960f None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:59.624431 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:59.625020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.625297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.625594 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.626248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:06:59.626547 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:59.628182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:59.639387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1359/2711] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:06:59.643616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56aca8-108a-42ee-bf62-4d2f0156aefa None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:59.652706 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56aca8-108a-42ee-bf62-4d2f0156aefa None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:59.657270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dbd6c-0f15-4d41-b28f-e45853ef0455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:59.657855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dbd6c-0f15-4d41-b28f-e45853ef0455 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.658133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dbd6c-0f15-4d41-b28f-e45853ef0455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.658403 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dbd6c-0f15-4d41-b28f-e45853ef0455 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.659059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dbd6c-0f15-4d41-b28f-e45853ef0455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:59.660936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dbd6c-0f15-4d41-b28f-e45853ef0455 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:59.691726 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1353/2712] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:06:59 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:59.692928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:59.702712 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7a332-8d02-4faf-89c0-6f2ee59f1010 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:59.715838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:59.769803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1c3ae-8657-44e6-9939-fc33ed9b174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:59.770490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1c3ae-8657-44e6-9939-fc33ed9b174c admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.770743 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1c3ae-8657-44e6-9939-fc33ed9b174c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.770987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1c3ae-8657-44e6-9939-fc33ed9b174c admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.771628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1c3ae-8657-44e6-9939-fc33ed9b174c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:59.776425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1c3ae-8657-44e6-9939-fc33ed9b174c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:59.799356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1360/2713] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:06:59 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:06:59.803139 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dbd6c-0f15-4d41-b28f-e45853ef0455 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:59.814135 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dbd6c-0f15-4d41-b28f-e45853ef0455 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:59.818684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b9a5a-824e-4b19-8a5c-3716c6921586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:59.819311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b9a5a-824e-4b19-8a5c-3716c6921586 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.819587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b9a5a-824e-4b19-8a5c-3716c6921586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.819855 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b9a5a-824e-4b19-8a5c-3716c6921586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.820537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b9a5a-824e-4b19-8a5c-3716c6921586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:59.822094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b9a5a-824e-4b19-8a5c-3716c6921586 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:59.831946 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1354/2714] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:59.835789 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1c3ae-8657-44e6-9939-fc33ed9b174c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:59.845685 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1c3ae-8657-44e6-9939-fc33ed9b174c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:59.850450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ca17-8eb9-44f0-98bf-a6bf30ff5ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:59.851088 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ca17-8eb9-44f0-98bf-a6bf30ff5ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.851374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ca17-8eb9-44f0-98bf-a6bf30ff5ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.852183 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ca17-8eb9-44f0-98bf-a6bf30ff5ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.852829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ca17-8eb9-44f0-98bf-a6bf30ff5ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:59.854688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ca17-8eb9-44f0-98bf-a6bf30ff5ac5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:59.869945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1361/2715] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:06:59.871274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b9a5a-824e-4b19-8a5c-3716c6921586 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:59.882027 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b9a5a-824e-4b19-8a5c-3716c6921586 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:59.886881 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34484b-cb83-4cc3-a16d-f6d747b3c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:59.887554 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34484b-cb83-4cc3-a16d-f6d747b3c422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.887836 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34484b-cb83-4cc3-a16d-f6d747b3c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.888106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34484b-cb83-4cc3-a16d-f6d747b3c422 admin admin] PATH_INFO: `/v3/projects/8b508e99393546bca2e9fde31b5bd225/users/f3dc3378f30849e4a716540d91a39696/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.902699 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34484b-cb83-4cc3-a16d-f6d747b3c422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b508e99393546bca2e9fde31b5bd225, user_id=f3dc3378f30849e4a716540d91a39696, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:59.905394 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34484b-cb83-4cc3-a16d-f6d747b3c422 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:59.921077 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1355/2716] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:06:59 2022] PUT /identity/v3/projects/8b508e99393546bca2e9fde31b5bd225/users/f3dc3378f30849e4a716540d91a39696/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:06:59.934059 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874c88c-d51a-4234-b93a-73591adfbcc2 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:59.934384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874c88c-d51a-4234-b93a-73591adfbcc2 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:59.934705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874c88c-d51a-4234-b93a-73591adfbcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:59.969069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a874c88c-d51a-4234-b93a-73591adfbcc2 None None] MFA Rules not processed for user `f3dc3378f30849e4a716540d91a39696`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:59.970328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a874c88c-d51a-4234-b93a-73591adfbcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:00.054974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1362/2717] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:00.059591 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34484b-cb83-4cc3-a16d-f6d747b3c422 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:00.105853 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:00.145631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34484b-cb83-4cc3-a16d-f6d747b3c422 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:00.152781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e78255-0286-4560-baf1-0f0daf247887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:00.154499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78255-0286-4560-baf1-0f0daf247887 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:00.154930 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78255-0286-4560-baf1-0f0daf247887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:00.155643 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78255-0286-4560-baf1-0f0daf247887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:00.156892 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e78255-0286-4560-baf1-0f0daf247887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:00.160382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e78255-0286-4560-baf1-0f0daf247887 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:00.205675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1356/2718] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:00 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:00.209879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874c88c-d51a-4234-b93a-73591adfbcc2 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:00.223043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874c88c-d51a-4234-b93a-73591adfbcc2 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:00.310187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d38b1d-1906-4a3a-a272-206a4581d43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:00.310831 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d38b1d-1906-4a3a-a272-206a4581d43d admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:00.311243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d38b1d-1906-4a3a-a272-206a4581d43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:00.311569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d38b1d-1906-4a3a-a272-206a4581d43d admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:00.312208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d38b1d-1906-4a3a-a272-206a4581d43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:00.314144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d38b1d-1906-4a3a-a272-206a4581d43d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:00.346740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1363/2719] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:00 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:07:00.347919 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e78255-0286-4560-baf1-0f0daf247887 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:00.362860 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e78255-0286-4560-baf1-0f0daf247887 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:00.368885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ac80e-25b6-46fa-ad4d-a30d8428627b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:00.369788 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ac80e-25b6-46fa-ad4d-a30d8428627b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:00.370170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ac80e-25b6-46fa-ad4d-a30d8428627b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:00.370727 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ac80e-25b6-46fa-ad4d-a30d8428627b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:00.371617 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ac80e-25b6-46fa-ad4d-a30d8428627b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:00.373458 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ac80e-25b6-46fa-ad4d-a30d8428627b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:00.387611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1357/2720] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:00.392236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d38b1d-1906-4a3a-a272-206a4581d43d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:00.408532 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d38b1d-1906-4a3a-a272-206a4581d43d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:00.420228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaab830-20d5-4630-858b-9db6791b72b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:00.421225 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaab830-20d5-4630-858b-9db6791b72b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:00.423073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaab830-20d5-4630-858b-9db6791b72b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:00.423525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaab830-20d5-4630-858b-9db6791b72b7 admin admin] PATH_INFO: `/v3/projects/6916623c14c940d3a19f47dac2862d8d/users/21b7eae561ae42b9936d6f5bc769431a/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:00.449759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:00.460621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaab830-20d5-4630-858b-9db6791b72b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6916623c14c940d3a19f47dac2862d8d, user_id=21b7eae561ae42b9936d6f5bc769431a, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:00.464756 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaab830-20d5-4630-858b-9db6791b72b7 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:00.484672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1364/2721] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:00 2022] PUT /identity/v3/projects/6916623c14c940d3a19f47dac2862d8d/users/21b7eae561ae42b9936d6f5bc769431a/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:00.488921 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ac80e-25b6-46fa-ad4d-a30d8428627b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:00.500712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ac80e-25b6-46fa-ad4d-a30d8428627b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:00.588399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991b6e2-2956-4707-8bc4-d7257f942678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:00.589028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991b6e2-2956-4707-8bc4-d7257f942678 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:00.589311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991b6e2-2956-4707-8bc4-d7257f942678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:00.589585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991b6e2-2956-4707-8bc4-d7257f942678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:00.590223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991b6e2-2956-4707-8bc4-d7257f942678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:00.592095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991b6e2-2956-4707-8bc4-d7257f942678 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:00.602003 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1358/2722] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:00.606213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaab830-20d5-4630-858b-9db6791b72b7 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:00.616515 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaab830-20d5-4630-858b-9db6791b72b7 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:00.621155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:00.621802 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:00.622082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:00.622356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] PATH_INFO: `/v3/projects/6916623c14c940d3a19f47dac2862d8d/users/21b7eae561ae42b9936d6f5bc769431a/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:00.624281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6916623c14c940d3a19f47dac2862d8d, user_id=21b7eae561ae42b9936d6f5bc769431a, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:00.626919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:00.637866 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1365/2723] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:00 2022] PUT /identity/v3/projects/6916623c14c940d3a19f47dac2862d8d/users/21b7eae561ae42b9936d6f5bc769431a/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:00.647792 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200dda-7e8f-49b4-b9a1-d533ef9abb72 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:00.648080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200dda-7e8f-49b4-b9a1-d533ef9abb72 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:00.648356 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200dda-7e8f-49b4-b9a1-d533ef9abb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:00.684500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-54200dda-7e8f-49b4-b9a1-d533ef9abb72 None None] MFA Rules not processed for user `21b7eae561ae42b9936d6f5bc769431a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:00.686029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-54200dda-7e8f-49b4-b9a1-d533ef9abb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:00.756362 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1359/2724] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:00.768259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:00.770060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:00.832748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf63e-56ca-41b2-a604-8ddf974f3ddd admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:00.836427 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd30f4-2355-4318-8b09-68663e5b2034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:00.837037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd30f4-2355-4318-8b09-68663e5b2034 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:00.837316 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd30f4-2355-4318-8b09-68663e5b2034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:00.838035 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd30f4-2355-4318-8b09-68663e5b2034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:00.846624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd30f4-2355-4318-8b09-68663e5b2034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:00.848092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd30f4-2355-4318-8b09-68663e5b2034 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:00.857961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1366/2725] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:00.931400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54200dda-7e8f-49b4-b9a1-d533ef9abb72 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:00.973253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:00.998961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54200dda-7e8f-49b4-b9a1-d533ef9abb72 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:01.004450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:01.005304 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:01.005695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:01.005982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] PATH_INFO: `/v3/users/f3dc3378f30849e4a716540d91a39696` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:01.007402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dc3378f30849e4a716540d91a39696)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:01.010632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:01.062976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dc3378f30849e4a716540d91a39696'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.069829 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dc3378f30849e4a716540d91a39696'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.070538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dc3378f30849e4a716540d91a39696'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.071166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dc3378f30849e4a716540d91a39696'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.079261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dc3378f30849e4a716540d91a39696'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.096909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1360/2726] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:00 2022] DELETE /identity/v3/users/f3dc3378f30849e4a716540d91a39696 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:01.101371 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7b8e-924f-41fa-a638-725f1568cd98 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:01.102265 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7b8e-924f-41fa-a638-725f1568cd98 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:01.102562 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7b8e-924f-41fa-a638-725f1568cd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:01.122200 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-e4dd7b8e-924f-41fa-a638-725f1568cd98 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:01.123476 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e4dd7b8e-924f-41fa-a638-725f1568cd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:01.186795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1367/2727] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:01.196632 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:01.199480 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:01.268282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf94ba-fe03-445b-b99a-cd6a4c872d86 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:01.271825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967706a-d2fc-4faa-a7fc-d6216d2ef0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:01.272426 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967706a-d2fc-4faa-a7fc-d6216d2ef0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:01.272705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967706a-d2fc-4faa-a7fc-d6216d2ef0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:01.272976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967706a-d2fc-4faa-a7fc-d6216d2ef0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:01.284391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967706a-d2fc-4faa-a7fc-d6216d2ef0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:01.285828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967706a-d2fc-4faa-a7fc-d6216d2ef0c4 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:01.296424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1361/2728] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:01.600818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd7b8e-924f-41fa-a638-725f1568cd98 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:01.602953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e4dd7b8e-924f-41fa-a638-725f1568cd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:01.625588 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd7b8e-924f-41fa-a638-725f1568cd98 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:01.631192 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:01.632166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:01.632794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:01.633327 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] PATH_INFO: `/v3/projects/8b508e99393546bca2e9fde31b5bd225` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:01.634697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b508e99393546bca2e9fde31b5bd225)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:01.636885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:01.690907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b508e99393546bca2e9fde31b5bd225', 'request_id': 'req-68724dce-14ef-4662-b15f-525b2644887f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.700569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b508e99393546bca2e9fde31b5bd225', 'request_id': 'req-68724dce-14ef-4662-b15f-525b2644887f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.707890 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b508e99393546bca2e9fde31b5bd225', 'request_id': 'req-68724dce-14ef-4662-b15f-525b2644887f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.719679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] The token cache is being invalidate because project 8b508e99393546bca2e9fde31b5bd225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:01.719972 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.720684 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.722240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1368/2729] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:01 2022] DELETE /identity/v3/projects/8b508e99393546bca2e9fde31b5bd225 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:01.726302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967706a-d2fc-4faa-a7fc-d6216d2ef0c4 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:01.737212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967706a-d2fc-4faa-a7fc-d6216d2ef0c4 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:01.794089 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:01.801442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:01.802130 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:01.802666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:01.803091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] PATH_INFO: `/v3/users/21b7eae561ae42b9936d6f5bc769431a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:01.804430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21b7eae561ae42b9936d6f5bc769431a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:01.807260 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:01.857022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21b7eae561ae42b9936d6f5bc769431a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.864096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21b7eae561ae42b9936d6f5bc769431a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.864896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21b7eae561ae42b9936d6f5bc769431a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.865495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21b7eae561ae42b9936d6f5bc769431a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.873182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b9523851-99a7-4160-b37e-c85b44f9e962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21b7eae561ae42b9936d6f5bc769431a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:01.896745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1362/2730] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:01 2022] DELETE /identity/v3/users/21b7eae561ae42b9936d6f5bc769431a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:02.198232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:02.199895 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:02.211766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:02.216132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68724dce-14ef-4662-b15f-525b2644887f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:02.283055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:02.283775 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.284130 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.284471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] PATH_INFO: `/v3/projects/6916623c14c940d3a19f47dac2862d8d` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.285687 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6916623c14c940d3a19f47dac2862d8d)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:02.287683 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:02.343285 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6916623c14c940d3a19f47dac2862d8d', 'request_id': 'req-0a904d15-c608-4b20-b699-b6c72e357d07'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:02.351637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6916623c14c940d3a19f47dac2862d8d', 'request_id': 'req-0a904d15-c608-4b20-b699-b6c72e357d07'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:02.358367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6916623c14c940d3a19f47dac2862d8d', 'request_id': 'req-0a904d15-c608-4b20-b699-b6c72e357d07'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:02.371141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] The token cache is being invalidate because project 6916623c14c940d3a19f47dac2862d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:02.371435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:02.372077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-0a904d15-c608-4b20-b699-b6c72e357d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:02.373553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1369/2731] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:02 2022] DELETE /identity/v3/projects/6916623c14c940d3a19f47dac2862d8d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:02.378381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1148-bc26-414f-8939-5a98401e3678 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.378666 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1148-bc26-414f-8939-5a98401e3678 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.379113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1148-bc26-414f-8939-5a98401e3678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.417311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-913d1148-bc26-414f-8939-5a98401e3678 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:02.419174 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-913d1148-bc26-414f-8939-5a98401e3678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:02.503216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1363/2732] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:02.504410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc1d90-965a-4d2f-ac3e-108f8296173e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.504637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc1d90-965a-4d2f-ac3e-108f8296173e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.504911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc1d90-965a-4d2f-ac3e-108f8296173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.537312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bcbc1d90-965a-4d2f-ac3e-108f8296173e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:02.539610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bcbc1d90-965a-4d2f-ac3e-108f8296173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:02.554231 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1370/2733] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:02.560063 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d1148-bc26-414f-8939-5a98401e3678 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:02.571355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d1148-bc26-414f-8939-5a98401e3678 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:02.580962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:02.581720 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.582031 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.582307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.583074 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:02.583407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:02.585096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:02.598495 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1364/2734] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:02.608581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc1d90-965a-4d2f-ac3e-108f8296173e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:02.621608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc1d90-965a-4d2f-ac3e-108f8296173e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:02.626941 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8ca52-1091-428b-a044-965bab033c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:02.627553 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8ca52-1091-428b-a044-965bab033c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.627834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8ca52-1091-428b-a044-965bab033c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.628133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8ca52-1091-428b-a044-965bab033c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.628765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8ca52-1091-428b-a044-965bab033c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:02.630666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8ca52-1091-428b-a044-965bab033c70 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:02.658545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1371/2735] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:02 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:02.663068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:02.673769 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3cd36-7368-4b2a-b582-d98d184536d7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:02.734280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea541e-b99c-44f5-b972-66239d646cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:02.734960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea541e-b99c-44f5-b972-66239d646cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.735177 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea541e-b99c-44f5-b972-66239d646cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.735451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea541e-b99c-44f5-b972-66239d646cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.736071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea541e-b99c-44f5-b972-66239d646cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:02.737967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea541e-b99c-44f5-b972-66239d646cd2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:02.748443 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:02.760605 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1365/2736] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:02 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:07:02.764865 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8ca52-1091-428b-a044-965bab033c70 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:02.774983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8ca52-1091-428b-a044-965bab033c70 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:02.779582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fe3f6-83a3-4678-afe1-7f3382262a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:02.780153 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fe3f6-83a3-4678-afe1-7f3382262a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.780433 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fe3f6-83a3-4678-afe1-7f3382262a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.780706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fe3f6-83a3-4678-afe1-7f3382262a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.781389 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fe3f6-83a3-4678-afe1-7f3382262a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:02.783092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fe3f6-83a3-4678-afe1-7f3382262a35 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:02.793418 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1372/2737] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:02.798399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea541e-b99c-44f5-b972-66239d646cd2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:02.812603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea541e-b99c-44f5-b972-66239d646cd2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:02.817541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf9b9a-4ffd-4305-a4da-7f14220c455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:02.818186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9b9a-4ffd-4305-a4da-7f14220c455f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.818469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9b9a-4ffd-4305-a4da-7f14220c455f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.818745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9b9a-4ffd-4305-a4da-7f14220c455f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.819424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf9b9a-4ffd-4305-a4da-7f14220c455f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:02.821033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf9b9a-4ffd-4305-a4da-7f14220c455f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:02.832071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1366/2738] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:07:02.836332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fe3f6-83a3-4678-afe1-7f3382262a35 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:02.846180 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fe3f6-83a3-4678-afe1-7f3382262a35 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:02.850761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:02.851438 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.852250 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.852526 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] PATH_INFO: `/v3/projects/6aee00de8cb64163b1fb3bb39f3daabe/users/f1e8ba8a07004b9382707eb71eb7240d/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.866626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aee00de8cb64163b1fb3bb39f3daabe, user_id=f1e8ba8a07004b9382707eb71eb7240d, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:02.869864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:02.882340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1373/2739] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:02 2022] PUT /identity/v3/projects/6aee00de8cb64163b1fb3bb39f3daabe/users/f1e8ba8a07004b9382707eb71eb7240d/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:02.947783 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb38277-836b-45e4-8dec-b6e67a6a56cb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:02.948071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb38277-836b-45e4-8dec-b6e67a6a56cb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:02.948344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb38277-836b-45e4-8dec-b6e67a6a56cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:02.976152 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fbb38277-836b-45e4-8dec-b6e67a6a56cb None None] MFA Rules not processed for user `f1e8ba8a07004b9382707eb71eb7240d`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:02.977951 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fbb38277-836b-45e4-8dec-b6e67a6a56cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:03.046970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1367/2740] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:03.057810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:03.059520 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:03.094014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:03.119031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cb498-6d63-4019-a564-7925f48d9340 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:03.122653 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef315ab7-9daa-4d02-a507-72841152ead0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:03.123286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315ab7-9daa-4d02-a507-72841152ead0 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:03.123569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315ab7-9daa-4d02-a507-72841152ead0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:03.123841 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315ab7-9daa-4d02-a507-72841152ead0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:03.132060 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef315ab7-9daa-4d02-a507-72841152ead0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:03.133478 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef315ab7-9daa-4d02-a507-72841152ead0 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:03.142038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1374/2741] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:04.505182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2feb9-1241-4d4c-b987-98b0674bc94f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:04.505654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2feb9-1241-4d4c-b987-98b0674bc94f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:04.505756 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2feb9-1241-4d4c-b987-98b0674bc94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:04.525952 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0da2feb9-1241-4d4c-b987-98b0674bc94f None None] MFA Rules not processed for user `ce6a1b5ca7584f90bfe83b73aff1335a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:04.527180 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0da2feb9-1241-4d4c-b987-98b0674bc94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:04.603252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1368/2742] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:13.138198 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=12 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:13.394234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ff2c-5903-4f42-810f-63365fbe229a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:13.394607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ff2c-5903-4f42-810f-63365fbe229a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:13.394988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ff2c-5903-4f42-810f-63365fbe229a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:13.425421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bda9ff2c-5903-4f42-810f-63365fbe229a None None] MFA Rules not processed for user `0243f437de8b439984375303ff49f9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:13.426703 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bda9ff2c-5903-4f42-810f-63365fbe229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:13.507018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1375/2743] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:14.592830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=1,keystone:SELECT=64 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:14.970373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2feb9-1241-4d4c-b987-98b0674bc94f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:15.102787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2feb9-1241-4d4c-b987-98b0674bc94f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:15.108038 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:15.108754 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:15.109136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:15.109512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] PATH_INFO: `/v3/users/f1e8ba8a07004b9382707eb71eb7240d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:15.110939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e8ba8a07004b9382707eb71eb7240d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:15.114050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:15.187181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e8ba8a07004b9382707eb71eb7240d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:15.214068 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e8ba8a07004b9382707eb71eb7240d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:15.215500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e8ba8a07004b9382707eb71eb7240d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:15.216133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e8ba8a07004b9382707eb71eb7240d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:15.237058 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e8ba8a07004b9382707eb71eb7240d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:15.281897 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1369/2744] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:14 2022] DELETE /identity/v3/users/f1e8ba8a07004b9382707eb71eb7240d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:15.286425 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8139c10-e143-4947-85aa-9934fb08d90d None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:15.286954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8139c10-e143-4947-85aa-9934fb08d90d None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:15.287054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8139c10-e143-4947-85aa-9934fb08d90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:15.399850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a8139c10-e143-4947-85aa-9934fb08d90d None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:15.404618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a8139c10-e143-4947-85aa-9934fb08d90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:15.498095 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1376/2745] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:15.557237 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:15.558724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:15.607057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:15.644446 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57987c7-a6e2-4e48-9a48-13c5de1625f7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:15.648903 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1ccf-bec2-455a-928e-d63f5ec83bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:15.649515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1ccf-bec2-455a-928e-d63f5ec83bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:15.650104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1ccf-bec2-455a-928e-d63f5ec83bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:15.650104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1ccf-bec2-455a-928e-d63f5ec83bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:15.660679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d1ccf-bec2-455a-928e-d63f5ec83bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:15.662220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d1ccf-bec2-455a-928e-d63f5ec83bb9 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:15.673891 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1370/2746] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:16.103110 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8139c10-e143-4947-85aa-9934fb08d90d None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:16.105252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a8139c10-e143-4947-85aa-9934fb08d90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:16.118178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8139c10-e143-4947-85aa-9934fb08d90d None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:16.135181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:16.136937 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:16.137380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:16.137810 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] PATH_INFO: `/v3/projects/6aee00de8cb64163b1fb3bb39f3daabe` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:16.139459 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aee00de8cb64163b1fb3bb39f3daabe)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:16.143137 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:16.218139 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aee00de8cb64163b1fb3bb39f3daabe', 'request_id': 'req-3b7ce6c6-4888-4532-af0a-d680f0231697'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:16.229206 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aee00de8cb64163b1fb3bb39f3daabe', 'request_id': 'req-3b7ce6c6-4888-4532-af0a-d680f0231697'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:16.230692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:16.237716 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aee00de8cb64163b1fb3bb39f3daabe', 'request_id': 'req-3b7ce6c6-4888-4532-af0a-d680f0231697'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:16.255022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] The token cache is being invalidate because project 6aee00de8cb64163b1fb3bb39f3daabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:16.255765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:16.257089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-3b7ce6c6-4888-4532-af0a-d680f0231697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:16.258927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1377/2747] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:16 2022] DELETE /identity/v3/projects/6aee00de8cb64163b1fb3bb39f3daabe => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:16.263637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df2ccd-92c0-4c99-baae-69a6d2b1c6f6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:16.264281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df2ccd-92c0-4c99-baae-69a6d2b1c6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:16.265087 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df2ccd-92c0-4c99-baae-69a6d2b1c6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:16.405960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e6df2ccd-92c0-4c99-baae-69a6d2b1c6f6 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:16.407775 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6df2ccd-92c0-4c99-baae-69a6d2b1c6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:16.569625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1371/2748] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:16.577436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee10159-ea06-4e98-a4b7-6d4db64a5bbe None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:16.578436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee10159-ea06-4e98-a4b7-6d4db64a5bbe None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:16.578883 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee10159-ea06-4e98-a4b7-6d4db64a5bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:16.628289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-dee10159-ea06-4e98-a4b7-6d4db64a5bbe None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:16.629967 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-dee10159-ea06-4e98-a4b7-6d4db64a5bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:16.649298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1378/2749] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:16.654221 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df2ccd-92c0-4c99-baae-69a6d2b1c6f6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:16.668880 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df2ccd-92c0-4c99-baae-69a6d2b1c6f6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:16.685512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:16.686447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:16.687002 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:16.687435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:16.688284 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:16.688767 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:16.690782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:16.716428 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1372/2750] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:16.717581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee10159-ea06-4e98-a4b7-6d4db64a5bbe None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:16.736922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee10159-ea06-4e98-a4b7-6d4db64a5bbe None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:16.744871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afc0d1-f4cb-4262-8b9a-88a5915eacee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:16.746014 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afc0d1-f4cb-4262-8b9a-88a5915eacee admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:16.747046 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afc0d1-f4cb-4262-8b9a-88a5915eacee admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:16.748070 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afc0d1-f4cb-4262-8b9a-88a5915eacee admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:16.749004 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afc0d1-f4cb-4262-8b9a-88a5915eacee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:16.752802 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afc0d1-f4cb-4262-8b9a-88a5915eacee admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:16.817071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1379/2751] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:16 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:16.818762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:16.835738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ab4a-fc58-473e-8796-8d8bd44eed8b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:16.912218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c676616-d9bd-4a0d-b958-6b99d5566817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:16.913718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c676616-d9bd-4a0d-b958-6b99d5566817 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:16.914192 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c676616-d9bd-4a0d-b958-6b99d5566817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:16.914470 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c676616-d9bd-4a0d-b958-6b99d5566817 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:16.915161 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c676616-d9bd-4a0d-b958-6b99d5566817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:16.917187 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c676616-d9bd-4a0d-b958-6b99d5566817 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:16.943928 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1373/2752] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:16 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:07:16.947637 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afc0d1-f4cb-4262-8b9a-88a5915eacee admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:16.960321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afc0d1-f4cb-4262-8b9a-88a5915eacee admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:16.965217 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a0c0b-3035-4344-9439-64012dff39bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:16.965934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0c0b-3035-4344-9439-64012dff39bd admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:16.966320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0c0b-3035-4344-9439-64012dff39bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:16.966694 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0c0b-3035-4344-9439-64012dff39bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:16.967471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a0c0b-3035-4344-9439-64012dff39bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:16.969372 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a0c0b-3035-4344-9439-64012dff39bd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:17.002197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1380/2753] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:17.016203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c676616-d9bd-4a0d-b958-6b99d5566817 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:17.029132 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c676616-d9bd-4a0d-b958-6b99d5566817 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:17.036901 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa836468-7ca1-4012-add8-dc20795f163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:17.038537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa836468-7ca1-4012-add8-dc20795f163a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:17.039136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa836468-7ca1-4012-add8-dc20795f163a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:17.039644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa836468-7ca1-4012-add8-dc20795f163a admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:17.045540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa836468-7ca1-4012-add8-dc20795f163a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:17.050970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa836468-7ca1-4012-add8-dc20795f163a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:17.084576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1374/2754] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:17.088211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a0c0b-3035-4344-9439-64012dff39bd admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:17.099342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a0c0b-3035-4344-9439-64012dff39bd admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:17.104954 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:17.105787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:17.106219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:17.106593 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] PATH_INFO: `/v3/projects/9fd4c00a2a834e6bbb76de37d1fabad3/users/88ad5f2c6b0e4c6a8aee91a97bc96dbb/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:17.123234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd4c00a2a834e6bbb76de37d1fabad3, user_id=88ad5f2c6b0e4c6a8aee91a97bc96dbb, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:17.126054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:17.153204 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1381/2755] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:17 2022] PUT /identity/v3/projects/9fd4c00a2a834e6bbb76de37d1fabad3/users/88ad5f2c6b0e4c6a8aee91a97bc96dbb/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:17.158764 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05faf9-5928-434b-b575-6e8154fd520f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:17.159939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05faf9-5928-434b-b575-6e8154fd520f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:17.160449 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05faf9-5928-434b-b575-6e8154fd520f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:17.180671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:17.199085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bb05faf9-5928-434b-b575-6e8154fd520f None None] MFA Rules not processed for user `88ad5f2c6b0e4c6a8aee91a97bc96dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:17.200566 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bb05faf9-5928-434b-b575-6e8154fd520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:17.287631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1375/2756] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:17.299249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:17.301834 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:17.365037 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc5398-3a1d-414e-9bfc-e5cd6a6c2f32 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:17.371185 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b372c-fc59-4157-81cc-bfb5fb55f141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:17.371946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b372c-fc59-4157-81cc-bfb5fb55f141 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:17.373025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b372c-fc59-4157-81cc-bfb5fb55f141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:17.373407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b372c-fc59-4157-81cc-bfb5fb55f141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:17.384178 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b372c-fc59-4157-81cc-bfb5fb55f141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:17.385729 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b372c-fc59-4157-81cc-bfb5fb55f141 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:17.402549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1382/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:20.588559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05faf9-5928-434b-b575-6e8154fd520f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:20.668593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05faf9-5928-434b-b575-6e8154fd520f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:20.694204 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:20.695280 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:20.695721 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:20.696175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] PATH_INFO: `/v3/users/88ad5f2c6b0e4c6a8aee91a97bc96dbb` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:20.698106 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88ad5f2c6b0e4c6a8aee91a97bc96dbb)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:20.700361 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:20.752854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88ad5f2c6b0e4c6a8aee91a97bc96dbb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:20.760784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88ad5f2c6b0e4c6a8aee91a97bc96dbb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:20.764197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88ad5f2c6b0e4c6a8aee91a97bc96dbb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:20.765489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88ad5f2c6b0e4c6a8aee91a97bc96dbb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:20.776599 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88ad5f2c6b0e4c6a8aee91a97bc96dbb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:20.801507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1376/2758] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:20 2022] DELETE /identity/v3/users/88ad5f2c6b0e4c6a8aee91a97bc96dbb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:20.806759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e60ec-4d11-46c2-b813-bdd9f043ef56 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:20.808328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e60ec-4d11-46c2-b813-bdd9f043ef56 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:20.809400 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e60ec-4d11-46c2-b813-bdd9f043ef56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:20.842408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-026e60ec-4d11-46c2-b813-bdd9f043ef56 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:20.843734 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-026e60ec-4d11-46c2-b813-bdd9f043ef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:20.920949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1383/2759] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:20.930959 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:20.932934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:20.958105 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:21.011040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ec622-a740-48c9-8bc4-0eff1c9003b2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:21.015138 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6e9ea-4f6d-4d8e-b632-45f12b1762da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:21.015985 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6e9ea-4f6d-4d8e-b632-45f12b1762da service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:21.016461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6e9ea-4f6d-4d8e-b632-45f12b1762da service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:21.016728 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6e9ea-4f6d-4d8e-b632-45f12b1762da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:21.026351 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6e9ea-4f6d-4d8e-b632-45f12b1762da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:21.027856 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6e9ea-4f6d-4d8e-b632-45f12b1762da service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:21.037541 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1377/2760] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:21.313658 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e60ec-4d11-46c2-b813-bdd9f043ef56 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:21.315193 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-026e60ec-4d11-46c2-b813-bdd9f043ef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:21.329914 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e60ec-4d11-46c2-b813-bdd9f043ef56 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:21.336321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:21.336987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:21.337310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:21.338644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] PATH_INFO: `/v3/projects/9fd4c00a2a834e6bbb76de37d1fabad3` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:21.339808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd4c00a2a834e6bbb76de37d1fabad3)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:21.342329 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:21.364467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:21.407711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd4c00a2a834e6bbb76de37d1fabad3', 'request_id': 'req-e226793a-6b82-4c40-95c4-6f174af6203d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:21.422885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd4c00a2a834e6bbb76de37d1fabad3', 'request_id': 'req-e226793a-6b82-4c40-95c4-6f174af6203d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:21.431406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd4c00a2a834e6bbb76de37d1fabad3', 'request_id': 'req-e226793a-6b82-4c40-95c4-6f174af6203d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:21.451312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] The token cache is being invalidate because project 9fd4c00a2a834e6bbb76de37d1fabad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:21.451597 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:21.452750 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:21.455098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1384/2761] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:21 2022] DELETE /identity/v3/projects/9fd4c00a2a834e6bbb76de37d1fabad3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:21.501195 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6e9ea-4f6d-4d8e-b632-45f12b1762da service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:21.510791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6e9ea-4f6d-4d8e-b632-45f12b1762da service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:21.574137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:21.574854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:21.575301 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:21.575647 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:21.577538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:21.669420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:21.671657 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:21.684927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1378/2762] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:25.990864 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:25.992618 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:26.018161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226793a-6b82-4c40-95c4-6f174af6203d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:26.024018 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:26.025460 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:26.025737 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:26.026170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] PATH_INFO: `/v3/users/b051e414038b470691ee3e91b952a87d` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:26.027367 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b051e414038b470691ee3e91b952a87d)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:26.029414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:26.094081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b051e414038b470691ee3e91b952a87d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.106380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b051e414038b470691ee3e91b952a87d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.107151 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b051e414038b470691ee3e91b952a87d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.108142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b051e414038b470691ee3e91b952a87d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.119440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-b673e01e-f47d-4419-b2eb-d084b100e687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b051e414038b470691ee3e91b952a87d'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.139458 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1385/2763] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:25 2022] DELETE /identity/v3/users/b051e414038b470691ee3e91b952a87d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:26.367107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:26.381698 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621664ce-d2c8-4aa0-85dc-34c6f233a1c9 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:26.502500 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:26.503665 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:26.503960 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:26.504223 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] PATH_INFO: `/v3/projects/0156957151444f94be28a65b4b018156` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:26.506098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0156957151444f94be28a65b4b018156)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:26.508577 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:26.527492 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:26.621044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0156957151444f94be28a65b4b018156', 'request_id': 'req-022256e5-0e58-4507-bca5-b8c73ab510b0'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.636877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0156957151444f94be28a65b4b018156', 'request_id': 'req-022256e5-0e58-4507-bca5-b8c73ab510b0'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.645674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0156957151444f94be28a65b4b018156', 'request_id': 'req-022256e5-0e58-4507-bca5-b8c73ab510b0'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.659198 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] The token cache is being invalidate because project 0156957151444f94be28a65b4b018156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:26.659669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.660267 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-022256e5-0e58-4507-bca5-b8c73ab510b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:26.662043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1379/2764] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:26 2022] DELETE /identity/v3/projects/0156957151444f94be28a65b4b018156 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:26.667600 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df5b27-832f-4fc3-95f1-10d15b7cd795 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:26.668268 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df5b27-832f-4fc3-95f1-10d15b7cd795 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:26.668747 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df5b27-832f-4fc3-95f1-10d15b7cd795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:26.693305 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-65df5b27-832f-4fc3-95f1-10d15b7cd795 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:26.694662 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-65df5b27-832f-4fc3-95f1-10d15b7cd795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:26.755891 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1386/2765] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:26.763678 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804452b2-1f7c-45de-abe6-c56f51a61f64 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:26.763966 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804452b2-1f7c-45de-abe6-c56f51a61f64 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:26.764240 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804452b2-1f7c-45de-abe6-c56f51a61f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:26.785024 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-804452b2-1f7c-45de-abe6-c56f51a61f64 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:26.787185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-804452b2-1f7c-45de-abe6-c56f51a61f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:26.798536 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1380/2766] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:26.802545 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df5b27-832f-4fc3-95f1-10d15b7cd795 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:26.812776 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df5b27-832f-4fc3-95f1-10d15b7cd795 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:26.817481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:26.818086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:26.818363 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:26.818753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:26.819434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:26.819735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:26.821397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:26.833518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1387/2767] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:26.837938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804452b2-1f7c-45de-abe6-c56f51a61f64 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:26.848819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804452b2-1f7c-45de-abe6-c56f51a61f64 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:26.854557 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50a627-5a72-4980-bd15-30c2164d08e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:26.855148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50a627-5a72-4980-bd15-30c2164d08e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:26.855430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50a627-5a72-4980-bd15-30c2164d08e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:26.855700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50a627-5a72-4980-bd15-30c2164d08e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:26.856331 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50a627-5a72-4980-bd15-30c2164d08e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:26.858218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50a627-5a72-4980-bd15-30c2164d08e6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:26.887736 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1381/2768] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:26 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:26.891940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:26.902396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85987796-b20d-4964-bb9d-624fa9089dcd admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:26.906211 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:26.962045 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92494b03-6b6f-4dbe-8a46-e348b13ce933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:26.963104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92494b03-6b6f-4dbe-8a46-e348b13ce933 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:26.963382 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92494b03-6b6f-4dbe-8a46-e348b13ce933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:26.963817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92494b03-6b6f-4dbe-8a46-e348b13ce933 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:26.964454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92494b03-6b6f-4dbe-8a46-e348b13ce933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:26.966343 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92494b03-6b6f-4dbe-8a46-e348b13ce933 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:26.989692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1388/2769] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:26 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:07:26.993712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50a627-5a72-4980-bd15-30c2164d08e6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:27.003910 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50a627-5a72-4980-bd15-30c2164d08e6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:27.008559 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c82a1a-fd84-4356-b350-8667a51dfd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:27.009156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82a1a-fd84-4356-b350-8667a51dfd62 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:27.009440 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82a1a-fd84-4356-b350-8667a51dfd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:27.009713 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82a1a-fd84-4356-b350-8667a51dfd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:27.010342 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c82a1a-fd84-4356-b350-8667a51dfd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:27.011954 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c82a1a-fd84-4356-b350-8667a51dfd62 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:27.022145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1382/2770] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:27.026767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92494b03-6b6f-4dbe-8a46-e348b13ce933 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:27.036770 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92494b03-6b6f-4dbe-8a46-e348b13ce933 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:27.041979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea4b2e-efd7-4f0b-b99f-be33c00ca336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:27.042647 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea4b2e-efd7-4f0b-b99f-be33c00ca336 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:27.042958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea4b2e-efd7-4f0b-b99f-be33c00ca336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:27.043232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea4b2e-efd7-4f0b-b99f-be33c00ca336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:27.043877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea4b2e-efd7-4f0b-b99f-be33c00ca336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:27.045450 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea4b2e-efd7-4f0b-b99f-be33c00ca336 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:27.056284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1389/2771] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:27.060514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c82a1a-fd84-4356-b350-8667a51dfd62 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:27.070994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c82a1a-fd84-4356-b350-8667a51dfd62 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:27.075762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:27.076411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:27.076694 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:27.076970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] PATH_INFO: `/v3/projects/d7a6a489452a49399c80d7bd29a07031/users/e1cf6ac02a39435f95d0cb8ead6de1bb/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:27.093875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a6a489452a49399c80d7bd29a07031, user_id=e1cf6ac02a39435f95d0cb8ead6de1bb, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:27.096718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:27.110800 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1383/2772] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:27 2022] PUT /identity/v3/projects/d7a6a489452a49399c80d7bd29a07031/users/e1cf6ac02a39435f95d0cb8ead6de1bb/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:27.121494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf11c6-6316-4546-80a4-539870f7990c None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:27.121783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf11c6-6316-4546-80a4-539870f7990c None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:27.122061 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf11c6-6316-4546-80a4-539870f7990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:27.149495 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-2cdf11c6-6316-4546-80a4-539870f7990c None None] MFA Rules not processed for user `e1cf6ac02a39435f95d0cb8ead6de1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:27.151210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2cdf11c6-6316-4546-80a4-539870f7990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:27.213430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1390/2773] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:27.225104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:27.227229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:27.328485 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf74a19-99f7-4145-9270-b1490fa7e83c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:27.333940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a992ae-042b-45c6-ba95-cfadd0f05d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:27.334533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a992ae-042b-45c6-ba95-cfadd0f05d30 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:27.334814 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a992ae-042b-45c6-ba95-cfadd0f05d30 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:27.335115 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a992ae-042b-45c6-ba95-cfadd0f05d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:27.340247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:27.344474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a992ae-042b-45c6-ba95-cfadd0f05d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:27.346086 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a992ae-042b-45c6-ba95-cfadd0f05d30 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:27.356150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1384/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:36.151311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf11c6-6316-4546-80a4-539870f7990c None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.210863 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf11c6-6316-4546-80a4-539870f7990c None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.215698 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.216320 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.216607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.216879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] PATH_INFO: `/v3/users/e1cf6ac02a39435f95d0cb8ead6de1bb` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.217974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1cf6ac02a39435f95d0cb8ead6de1bb)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.219924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.236218 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:36.266558 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1cf6ac02a39435f95d0cb8ead6de1bb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.273849 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1cf6ac02a39435f95d0cb8ead6de1bb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.275112 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1cf6ac02a39435f95d0cb8ead6de1bb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.275730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1cf6ac02a39435f95d0cb8ead6de1bb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.283341 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1cf6ac02a39435f95d0cb8ead6de1bb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.301987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1391/2775] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:36 2022] DELETE /identity/v3/users/e1cf6ac02a39435f95d0cb8ead6de1bb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:36.307011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbddeb5-a998-4a16-b7c4-f0b4462c3585 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.307508 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbddeb5-a998-4a16-b7c4-f0b4462c3585 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.307624 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbddeb5-a998-4a16-b7c4-f0b4462c3585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.333804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6bbddeb5-a998-4a16-b7c4-f0b4462c3585 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:36.335253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6bbddeb5-a998-4a16-b7c4-f0b4462c3585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:36.416017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1385/2776] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:36.426291 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.428184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:36.524001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba4197-c4b6-45ef-b5e8-a78e44984879 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.527610 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270260d3-4ee1-4067-8443-8a3c1028d4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.528226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270260d3-4ee1-4067-8443-8a3c1028d4ee service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.528504 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270260d3-4ee1-4067-8443-8a3c1028d4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.528766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270260d3-4ee1-4067-8443-8a3c1028d4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.538778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270260d3-4ee1-4067-8443-8a3c1028d4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.540247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270260d3-4ee1-4067-8443-8a3c1028d4ee service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.550682 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1392/2777] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:36.816140 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbddeb5-a998-4a16-b7c4-f0b4462c3585 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.817552 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6bbddeb5-a998-4a16-b7c4-f0b4462c3585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:36.830360 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbddeb5-a998-4a16-b7c4-f0b4462c3585 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.835261 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.836121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.836410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.836681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] PATH_INFO: `/v3/projects/d7a6a489452a49399c80d7bd29a07031` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.838430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a6a489452a49399c80d7bd29a07031)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.840355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.889750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a6a489452a49399c80d7bd29a07031', 'request_id': 'req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.897512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a6a489452a49399c80d7bd29a07031', 'request_id': 'req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.904886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a6a489452a49399c80d7bd29a07031', 'request_id': 'req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.915496 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] The token cache is being invalidate because project d7a6a489452a49399c80d7bd29a07031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:36.915784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.916429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0d543b90-64f7-4a98-ba17-d29ae19d9ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:36.917895 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1386/2778] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:36 2022] DELETE /identity/v3/projects/d7a6a489452a49399c80d7bd29a07031 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:36.923063 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09231-9bf6-4703-9e4e-62ac7e6e0401 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.923350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09231-9bf6-4703-9e4e-62ac7e6e0401 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.923621 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09231-9bf6-4703-9e4e-62ac7e6e0401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.943705 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6af09231-9bf6-4703-9e4e-62ac7e6e0401 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:36.944889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6af09231-9bf6-4703-9e4e-62ac7e6e0401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.002566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1393/2779] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:37.010656 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870ff1e-ea68-49ee-9a6a-c0cfa87096f7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.010962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870ff1e-ea68-49ee-9a6a-c0cfa87096f7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.011245 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870ff1e-ea68-49ee-9a6a-c0cfa87096f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.031266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8870ff1e-ea68-49ee-9a6a-c0cfa87096f7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.032456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8870ff1e-ea68-49ee-9a6a-c0cfa87096f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.040697 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1387/2780] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:37.044798 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af09231-9bf6-4703-9e4e-62ac7e6e0401 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.055136 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af09231-9bf6-4703-9e4e-62ac7e6e0401 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.060245 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.060774 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.061049 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.061314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.061944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:37.062240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.063837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.075639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1394/2781] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:37.080070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870ff1e-ea68-49ee-9a6a-c0cfa87096f7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.089253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870ff1e-ea68-49ee-9a6a-c0cfa87096f7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.093869 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070dfdee-8179-4945-bce8-cd2a0c8d2e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.094457 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dfdee-8179-4945-bce8-cd2a0c8d2e43 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.094739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dfdee-8179-4945-bce8-cd2a0c8d2e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.095056 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dfdee-8179-4945-bce8-cd2a0c8d2e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.095680 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070dfdee-8179-4945-bce8-cd2a0c8d2e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.097571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070dfdee-8179-4945-bce8-cd2a0c8d2e43 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.124308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1388/2782] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:37.128271 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.137239 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:37.141256 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b4e9f-3743-4611-bd8c-073fad7a499c admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.197576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93160e52-5068-40bd-831d-176acd709c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.198726 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93160e52-5068-40bd-831d-176acd709c4d admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.198960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93160e52-5068-40bd-831d-176acd709c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.199228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93160e52-5068-40bd-831d-176acd709c4d admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.199843 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93160e52-5068-40bd-831d-176acd709c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.201738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93160e52-5068-40bd-831d-176acd709c4d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.222537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1395/2783] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:37.227070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070dfdee-8179-4945-bce8-cd2a0c8d2e43 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.237528 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070dfdee-8179-4945-bce8-cd2a0c8d2e43 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.242050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81da0-279d-47e0-b4b4-3571a0c35e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.242638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81da0-279d-47e0-b4b4-3571a0c35e28 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.242961 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81da0-279d-47e0-b4b4-3571a0c35e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.243235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81da0-279d-47e0-b4b4-3571a0c35e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.243866 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81da0-279d-47e0-b4b4-3571a0c35e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.245972 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81da0-279d-47e0-b4b4-3571a0c35e28 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.255640 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1389/2784] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:37.259453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93160e52-5068-40bd-831d-176acd709c4d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.269793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93160e52-5068-40bd-831d-176acd709c4d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.274432 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e785f-4121-40c3-a112-2503fb5a9823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.275054 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e785f-4121-40c3-a112-2503fb5a9823 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.275337 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e785f-4121-40c3-a112-2503fb5a9823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.275608 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e785f-4121-40c3-a112-2503fb5a9823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.276240 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e785f-4121-40c3-a112-2503fb5a9823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.277816 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e785f-4121-40c3-a112-2503fb5a9823 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.288155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1396/2785] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:37.292314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81da0-279d-47e0-b4b4-3571a0c35e28 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.302514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81da0-279d-47e0-b4b4-3571a0c35e28 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.307535 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.308160 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.308435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.308701 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] PATH_INFO: `/v3/projects/7e8e2ba9d0dd48edb38f781d5346812c/users/08b1d59aa84a4c4e8faffcc459b31350/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.323745 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8e2ba9d0dd48edb38f781d5346812c, user_id=08b1d59aa84a4c4e8faffcc459b31350, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.326373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.339067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1390/2786] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:37 2022] PUT /identity/v3/projects/7e8e2ba9d0dd48edb38f781d5346812c/users/08b1d59aa84a4c4e8faffcc459b31350/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:37.349101 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5d4e-62da-4543-8048-0bb31faaba0e None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.349390 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5d4e-62da-4543-8048-0bb31faaba0e None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.349666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5d4e-62da-4543-8048-0bb31faaba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.377066 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-810b5d4e-62da-4543-8048-0bb31faaba0e None None] MFA Rules not processed for user `08b1d59aa84a4c4e8faffcc459b31350`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.378325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-810b5d4e-62da-4543-8048-0bb31faaba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.441871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1397/2787] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:37.452576 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.454272 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.472054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:37.536761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5f74-5c69-44e7-95be-104559667d3f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.540939 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b115b0f-e894-4a30-8859-4e9d055e6b46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.542057 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115b0f-e894-4a30-8859-4e9d055e6b46 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.542334 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115b0f-e894-4a30-8859-4e9d055e6b46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.542604 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115b0f-e894-4a30-8859-4e9d055e6b46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.551182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b115b0f-e894-4a30-8859-4e9d055e6b46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.552593 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b115b0f-e894-4a30-8859-4e9d055e6b46 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.561871 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1391/2788] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:40.205412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b5d4e-62da-4543-8048-0bb31faaba0e None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:40.207034 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-810b5d4e-62da-4543-8048-0bb31faaba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:40.266670 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b5d4e-62da-4543-8048-0bb31faaba0e None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:40.270437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096ae1e-0e66-4900-8207-c54a3c39f4f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:40.271074 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ae1e-0e66-4900-8207-c54a3c39f4f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:40.271350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ae1e-0e66-4900-8207-c54a3c39f4f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:40.271616 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096ae1e-0e66-4900-8207-c54a3c39f4f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:40.273488 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2096ae1e-0e66-4900-8207-c54a3c39f4f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:40.302766 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:40.340626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ae1e-0e66-4900-8207-c54a3c39f4f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:40.342051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096ae1e-0e66-4900-8207-c54a3c39f4f6 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:40.352276 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1398/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:47.557347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=29 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:50.347983 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=17 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:51.276424 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b115b0f-e894-4a30-8859-4e9d055e6b46 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:51.339385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b115b0f-e894-4a30-8859-4e9d055e6b46 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:51.343811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:51.344428 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.344705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.344974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] PATH_INFO: `/v3/users/08b1d59aa84a4c4e8faffcc459b31350` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.346091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] RBAC: Authorizing `identity:delete_user(user_id=08b1d59aa84a4c4e8faffcc459b31350)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:51.348033 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:51.393882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08b1d59aa84a4c4e8faffcc459b31350'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.400711 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08b1d59aa84a4c4e8faffcc459b31350'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.401377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08b1d59aa84a4c4e8faffcc459b31350'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.402065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08b1d59aa84a4c4e8faffcc459b31350'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.410601 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08b1d59aa84a4c4e8faffcc459b31350'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.428359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1392/2790] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:51 2022] DELETE /identity/v3/users/08b1d59aa84a4c4e8faffcc459b31350 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:51.432793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce84425-6a6a-4f48-90cc-c7c066def465 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.433769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce84425-6a6a-4f48-90cc-c7c066def465 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.433885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce84425-6a6a-4f48-90cc-c7c066def465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.452907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-7ce84425-6a6a-4f48-90cc-c7c066def465 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:51.454166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7ce84425-6a6a-4f48-90cc-c7c066def465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:51.509947 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1399/2791] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:51.520137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:51.521522 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:51.577735 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48236d6-c6d0-4b69-a5ed-cb6defe5964a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:51.581396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034a0f-ccab-47a9-8fdd-bf9c95cd9407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:51.581992 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034a0f-ccab-47a9-8fdd-bf9c95cd9407 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.582270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034a0f-ccab-47a9-8fdd-bf9c95cd9407 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.582540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034a0f-ccab-47a9-8fdd-bf9c95cd9407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.592116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52034a0f-ccab-47a9-8fdd-bf9c95cd9407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:51.593527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52034a0f-ccab-47a9-8fdd-bf9c95cd9407 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:51.602882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1393/2792] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:07:51.867187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce84425-6a6a-4f48-90cc-c7c066def465 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:51.868636 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7ce84425-6a6a-4f48-90cc-c7c066def465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:51.881166 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce84425-6a6a-4f48-90cc-c7c066def465 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:51.885815 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:51.886452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.886735 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.887041 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] PATH_INFO: `/v3/projects/7e8e2ba9d0dd48edb38f781d5346812c` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.888161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8e2ba9d0dd48edb38f781d5346812c)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:51.890072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:51.937728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8e2ba9d0dd48edb38f781d5346812c', 'request_id': 'req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.945420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8e2ba9d0dd48edb38f781d5346812c', 'request_id': 'req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.952262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8e2ba9d0dd48edb38f781d5346812c', 'request_id': 'req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.962780 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] The token cache is being invalidate because project 7e8e2ba9d0dd48edb38f781d5346812c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:51.963095 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.963745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4b0ea1a3-89a2-45fc-ac3a-a1411f918180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.965328 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1400/2793] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:51 2022] DELETE /identity/v3/projects/7e8e2ba9d0dd48edb38f781d5346812c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:51.970122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d7e32-4cb8-4ac2-bdd1-3736cac54317 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.970396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d7e32-4cb8-4ac2-bdd1-3736cac54317 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.970672 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d7e32-4cb8-4ac2-bdd1-3736cac54317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.990514 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7a6d7e32-4cb8-4ac2-bdd1-3736cac54317 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:51.991761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7a6d7e32-4cb8-4ac2-bdd1-3736cac54317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:52.011041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:52.056938 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1394/2794] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:52.065727 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e948435-e758-43b8-ab95-d4d9d6362df4 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.066008 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e948435-e758-43b8-ab95-d4d9d6362df4 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.066312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e948435-e758-43b8-ab95-d4d9d6362df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.085374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3e948435-e758-43b8-ab95-d4d9d6362df4 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:52.086566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3e948435-e758-43b8-ab95-d4d9d6362df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:52.094761 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1401/2795] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:52.098825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d7e32-4cb8-4ac2-bdd1-3736cac54317 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.109091 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d7e32-4cb8-4ac2-bdd1-3736cac54317 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.113600 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.114190 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.114472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.114743 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.115417 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:52.115718 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.117298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.128612 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1395/2796] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:07:52.133058 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e948435-e758-43b8-ab95-d4d9d6362df4 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.142038 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e948435-e758-43b8-ab95-d4d9d6362df4 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.146738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d0f30-9de0-4689-a982-36281ff7e3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.147412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d0f30-9de0-4689-a982-36281ff7e3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.147686 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d0f30-9de0-4689-a982-36281ff7e3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.147951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d0f30-9de0-4689-a982-36281ff7e3b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.148632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d0f30-9de0-4689-a982-36281ff7e3b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.150596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d0f30-9de0-4689-a982-36281ff7e3b1 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.179405 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1402/2797] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:52 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:52.183830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.193615 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b9140-4f5b-40a2-b6d2-b66aa67757b0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.255598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c656db-7147-4df4-bcf0-b048387044ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.256218 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c656db-7147-4df4-bcf0-b048387044ce admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.256501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c656db-7147-4df4-bcf0-b048387044ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.256774 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c656db-7147-4df4-bcf0-b048387044ce admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.257407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c656db-7147-4df4-bcf0-b048387044ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.259464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c656db-7147-4df4-bcf0-b048387044ce admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.281095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1396/2798] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:52 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:52.285436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d0f30-9de0-4689-a982-36281ff7e3b1 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.296379 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d0f30-9de0-4689-a982-36281ff7e3b1 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.301397 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466d136-d319-4a0c-9cc2-47c7d918c85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.301988 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466d136-d319-4a0c-9cc2-47c7d918c85d admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.302267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466d136-d319-4a0c-9cc2-47c7d918c85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.302540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466d136-d319-4a0c-9cc2-47c7d918c85d admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.303227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466d136-d319-4a0c-9cc2-47c7d918c85d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.304999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466d136-d319-4a0c-9cc2-47c7d918c85d admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.315234 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1403/2799] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:52.318885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c656db-7147-4df4-bcf0-b048387044ce admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.328825 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c656db-7147-4df4-bcf0-b048387044ce admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.333291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4373870-9bd3-449a-a405-1f4780837896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.333877 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373870-9bd3-449a-a405-1f4780837896 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.334153 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373870-9bd3-449a-a405-1f4780837896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.334421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373870-9bd3-449a-a405-1f4780837896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.335095 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4373870-9bd3-449a-a405-1f4780837896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.336664 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4373870-9bd3-449a-a405-1f4780837896 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.348572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1397/2800] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:52.352582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466d136-d319-4a0c-9cc2-47c7d918c85d admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.363020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466d136-d319-4a0c-9cc2-47c7d918c85d admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.367675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fdf65-f37e-455d-990a-51258015fd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.368309 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fdf65-f37e-455d-990a-51258015fd74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.368589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fdf65-f37e-455d-990a-51258015fd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.368860 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fdf65-f37e-455d-990a-51258015fd74 admin admin] PATH_INFO: `/v3/projects/3a94e4542093475b860c74410bdafb14/users/3a3870f8f1904376bcbb21d6cc4a625e/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.384850 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fdf65-f37e-455d-990a-51258015fd74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a94e4542093475b860c74410bdafb14, user_id=3a3870f8f1904376bcbb21d6cc4a625e, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.388089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fdf65-f37e-455d-990a-51258015fd74 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.400657 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1404/2801] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:52 2022] PUT /identity/v3/projects/3a94e4542093475b860c74410bdafb14/users/3a3870f8f1904376bcbb21d6cc4a625e/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:52.411447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac67af-e99a-4d43-8eab-2e7b1da2f1bf None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.411738 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac67af-e99a-4d43-8eab-2e7b1da2f1bf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.412011 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac67af-e99a-4d43-8eab-2e7b1da2f1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.439278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-13ac67af-e99a-4d43-8eab-2e7b1da2f1bf None None] MFA Rules not processed for user `3a3870f8f1904376bcbb21d6cc4a625e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:52.440474 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-13ac67af-e99a-4d43-8eab-2e7b1da2f1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:52.503248 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1398/2802] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:52.507689 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fdf65-f37e-455d-990a-51258015fd74 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.529288 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:52.571142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fdf65-f37e-455d-990a-51258015fd74 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.576340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d9c07-3f3a-47e5-9921-d07a9d9526fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.576935 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d9c07-3f3a-47e5-9921-d07a9d9526fd admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.577216 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d9c07-3f3a-47e5-9921-d07a9d9526fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.577490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d9c07-3f3a-47e5-9921-d07a9d9526fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.578114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d9c07-3f3a-47e5-9921-d07a9d9526fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.580044 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d9c07-3f3a-47e5-9921-d07a9d9526fd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.606383 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1405/2803] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:52 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:52.610407 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac67af-e99a-4d43-8eab-2e7b1da2f1bf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.621053 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac67af-e99a-4d43-8eab-2e7b1da2f1bf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.632940 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:52.678828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa002a-0519-4f4d-8c3c-3f0c53422a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.679462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa002a-0519-4f4d-8c3c-3f0c53422a3e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.679741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa002a-0519-4f4d-8c3c-3f0c53422a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.680007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa002a-0519-4f4d-8c3c-3f0c53422a3e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.680628 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa002a-0519-4f4d-8c3c-3f0c53422a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.682525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa002a-0519-4f4d-8c3c-3f0c53422a3e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.703302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1399/2804] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:52 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:52.707219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d9c07-3f3a-47e5-9921-d07a9d9526fd admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.717829 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d9c07-3f3a-47e5-9921-d07a9d9526fd admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.722362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd000b-227b-481f-be78-7e96557c7016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.722981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd000b-227b-481f-be78-7e96557c7016 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.723267 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd000b-227b-481f-be78-7e96557c7016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.723537 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd000b-227b-481f-be78-7e96557c7016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.724172 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd000b-227b-481f-be78-7e96557c7016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.725751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd000b-227b-481f-be78-7e96557c7016 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.736118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1406/2805] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:52.739942 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa002a-0519-4f4d-8c3c-3f0c53422a3e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.750472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa002a-0519-4f4d-8c3c-3f0c53422a3e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.755197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd89f0f-ea8e-45ce-9748-ecf4a6a11d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.755837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd89f0f-ea8e-45ce-9748-ecf4a6a11d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.756120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd89f0f-ea8e-45ce-9748-ecf4a6a11d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.756395 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd89f0f-ea8e-45ce-9748-ecf4a6a11d01 admin admin] PATH_INFO: `/v3/projects/009236f1c3944644bac4e43b16cd5c89/users/672683a4f44342be99c90e386a9b222a/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.771641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd89f0f-ea8e-45ce-9748-ecf4a6a11d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009236f1c3944644bac4e43b16cd5c89, user_id=672683a4f44342be99c90e386a9b222a, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.774275 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd89f0f-ea8e-45ce-9748-ecf4a6a11d01 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.787199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1400/2806] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:52 2022] PUT /identity/v3/projects/009236f1c3944644bac4e43b16cd5c89/users/672683a4f44342be99c90e386a9b222a/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:52.791210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd000b-227b-481f-be78-7e96557c7016 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.802236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd000b-227b-481f-be78-7e96557c7016 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.858254 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674c3e7-055a-4ef1-8661-6e57fd2f91fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.858924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c3e7-055a-4ef1-8661-6e57fd2f91fa admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.859208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c3e7-055a-4ef1-8661-6e57fd2f91fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.859480 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c3e7-055a-4ef1-8661-6e57fd2f91fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.860111 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674c3e7-055a-4ef1-8661-6e57fd2f91fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.861692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674c3e7-055a-4ef1-8661-6e57fd2f91fa admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.871911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1407/2807] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:52.876308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd89f0f-ea8e-45ce-9748-ecf4a6a11d01 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.887013 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd89f0f-ea8e-45ce-9748-ecf4a6a11d01 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.891469 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.892558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.892684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.893032 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] PATH_INFO: `/v3/projects/009236f1c3944644bac4e43b16cd5c89/users/672683a4f44342be99c90e386a9b222a/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.896379 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009236f1c3944644bac4e43b16cd5c89, user_id=672683a4f44342be99c90e386a9b222a, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.901684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.912268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1401/2808] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:52 2022] PUT /identity/v3/projects/009236f1c3944644bac4e43b16cd5c89/users/672683a4f44342be99c90e386a9b222a/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:52.923009 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3575c7-ebd2-4429-850a-792531081e22 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.923297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3575c7-ebd2-4429-850a-792531081e22 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.923569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3575c7-ebd2-4429-850a-792531081e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.951720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6e3575c7-ebd2-4429-850a-792531081e22 None None] MFA Rules not processed for user `672683a4f44342be99c90e386a9b222a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:52.952907 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6e3575c7-ebd2-4429-850a-792531081e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:53.019000 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1408/2809] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:53.030090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:53.031810 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:53.092445 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89d57e-9869-45cf-b3e3-67537ea68a03 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:53.096170 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6cf52-ecea-441d-be4a-8d84eedd6323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:53.096762 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6cf52-ecea-441d-be4a-8d84eedd6323 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:53.097040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6cf52-ecea-441d-be4a-8d84eedd6323 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:53.097308 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6cf52-ecea-441d-be4a-8d84eedd6323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:53.106845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6cf52-ecea-441d-be4a-8d84eedd6323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:53.108402 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6cf52-ecea-441d-be4a-8d84eedd6323 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:53.118505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1402/2810] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:53.195778 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3575c7-ebd2-4429-850a-792531081e22 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:53.205342 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3575c7-ebd2-4429-850a-792531081e22 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:53.208976 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1217c-6c29-4bb4-bf4c-1a4add30d1d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:53.209573 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1217c-6c29-4bb4-bf4c-1a4add30d1d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:53.209848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1217c-6c29-4bb4-bf4c-1a4add30d1d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:53.210117 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1217c-6c29-4bb4-bf4c-1a4add30d1d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:53.222629 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:53.273651 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1217c-6c29-4bb4-bf4c-1a4add30d1d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:53.275122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1217c-6c29-4bb4-bf4c-1a4add30d1d1 service cinder] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:53.284680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1409/2811] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:56.015636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6cf52-ecea-441d-be4a-8d84eedd6323 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.077316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6cf52-ecea-441d-be4a-8d84eedd6323 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.081862 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.082532 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.082818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.083122 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] PATH_INFO: `/v3/users/3a3870f8f1904376bcbb21d6cc4a625e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.084236 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3870f8f1904376bcbb21d6cc4a625e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.086137 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.099372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:56.133625 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3870f8f1904376bcbb21d6cc4a625e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.141112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3870f8f1904376bcbb21d6cc4a625e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.141787 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3870f8f1904376bcbb21d6cc4a625e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.142380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3870f8f1904376bcbb21d6cc4a625e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.149798 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3870f8f1904376bcbb21d6cc4a625e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.167555 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1403/2812] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:56 2022] DELETE /identity/v3/users/3a3870f8f1904376bcbb21d6cc4a625e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:56.172247 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf307008-83cf-48cd-ab41-a49ba9182821 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.172706 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf307008-83cf-48cd-ab41-a49ba9182821 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.172832 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf307008-83cf-48cd-ab41-a49ba9182821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.191332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bf307008-83cf-48cd-ab41-a49ba9182821 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:56.192518 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bf307008-83cf-48cd-ab41-a49ba9182821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:56.250395 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1410/2813] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:56.260083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.261507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:56.317729 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd9134-093d-426e-8361-b7494921ea05 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.321336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dbdf5-35a3-4596-817e-37ee1e1f6d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.321934 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dbdf5-35a3-4596-817e-37ee1e1f6d74 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.322215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dbdf5-35a3-4596-817e-37ee1e1f6d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.322490 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dbdf5-35a3-4596-817e-37ee1e1f6d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.331811 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dbdf5-35a3-4596-817e-37ee1e1f6d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.333247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dbdf5-35a3-4596-817e-37ee1e1f6d74 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.343129 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1404/2814] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:56.609940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf307008-83cf-48cd-ab41-a49ba9182821 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.611384 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bf307008-83cf-48cd-ab41-a49ba9182821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:56.625642 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf307008-83cf-48cd-ab41-a49ba9182821 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.630529 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.631208 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.631494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.631768 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] PATH_INFO: `/v3/projects/3a94e4542093475b860c74410bdafb14` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.632998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a94e4542093475b860c74410bdafb14)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.634785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.682964 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a94e4542093475b860c74410bdafb14', 'request_id': 'req-7f0f11d6-9344-4ad2-82d8-641f795f75fd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.691298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a94e4542093475b860c74410bdafb14', 'request_id': 'req-7f0f11d6-9344-4ad2-82d8-641f795f75fd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.697794 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a94e4542093475b860c74410bdafb14', 'request_id': 'req-7f0f11d6-9344-4ad2-82d8-641f795f75fd'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.708032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] The token cache is being invalidate because project 3a94e4542093475b860c74410bdafb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:56.708314 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.708944 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.710406 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1411/2815] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:56 2022] DELETE /identity/v3/projects/3a94e4542093475b860c74410bdafb14 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:56.714252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dbdf5-35a3-4596-817e-37ee1e1f6d74 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.725186 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dbdf5-35a3-4596-817e-37ee1e1f6d74 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.786112 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.786740 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.787054 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.787324 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] PATH_INFO: `/v3/users/672683a4f44342be99c90e386a9b222a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.788537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=672683a4f44342be99c90e386a9b222a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.790410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.831421 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:56.843103 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '672683a4f44342be99c90e386a9b222a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.852848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '672683a4f44342be99c90e386a9b222a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.854214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '672683a4f44342be99c90e386a9b222a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.855355 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '672683a4f44342be99c90e386a9b222a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.864224 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98488138-6644-46c3-90eb-6eb8ed1084ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '672683a4f44342be99c90e386a9b222a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:56.882868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1405/2816] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:56 2022] DELETE /identity/v3/users/672683a4f44342be99c90e386a9b222a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:57.142374 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.143951 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.157469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f11d6-9344-4ad2-82d8-641f795f75fd admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.190827 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:57.215241 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.216505 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.216784 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.217052 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] PATH_INFO: `/v3/projects/009236f1c3944644bac4e43b16cd5c89` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.218187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=009236f1c3944644bac4e43b16cd5c89)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.220116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.265255 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009236f1c3944644bac4e43b16cd5c89', 'request_id': 'req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.272158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009236f1c3944644bac4e43b16cd5c89', 'request_id': 'req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.278382 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009236f1c3944644bac4e43b16cd5c89', 'request_id': 'req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.288720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] The token cache is being invalidate because project 009236f1c3944644bac4e43b16cd5c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:57.289007 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.289669 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-e6f5e7e3-f6dd-426a-bbbd-bb5495acccdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.291187 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1412/2817] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:57 2022] DELETE /identity/v3/projects/009236f1c3944644bac4e43b16cd5c89 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:57.295904 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb95a3-b181-4a75-b956-0b64938fbbd3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.296637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb95a3-b181-4a75-b956-0b64938fbbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.296739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb95a3-b181-4a75-b956-0b64938fbbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.317472 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-39bb95a3-b181-4a75-b956-0b64938fbbd3 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:57.318669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-39bb95a3-b181-4a75-b956-0b64938fbbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.376937 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1406/2818] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:57.384632 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032946e-1be5-41f4-99a0-79c1301d904a None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.384913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032946e-1be5-41f4-99a0-79c1301d904a None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.385181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032946e-1be5-41f4-99a0-79c1301d904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.408980 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-5032946e-1be5-41f4-99a0-79c1301d904a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:57.410196 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5032946e-1be5-41f4-99a0-79c1301d904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.419310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1413/2819] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:57.423996 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb95a3-b181-4a75-b956-0b64938fbbd3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.433840 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb95a3-b181-4a75-b956-0b64938fbbd3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.438883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.439501 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.439778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.440043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.440677 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:57.440978 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.442543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.454611 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1407/2820] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:57.459126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032946e-1be5-41f4-99a0-79c1301d904a None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.468295 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032946e-1be5-41f4-99a0-79c1301d904a None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.473163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3e4d5-37c6-4933-ab7c-626aa2f31e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.473791 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3e4d5-37c6-4933-ab7c-626aa2f31e72 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.474072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3e4d5-37c6-4933-ab7c-626aa2f31e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.474345 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3e4d5-37c6-4933-ab7c-626aa2f31e72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.475030 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3e4d5-37c6-4933-ab7c-626aa2f31e72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.476886 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3e4d5-37c6-4933-ab7c-626aa2f31e72 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.503311 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1414/2821] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:57.507406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.517742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10d16c-95ca-48a8-b19f-a1ff9434452b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.572515 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fa7f-8257-440a-8a06-94bfc8e88816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.573110 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fa7f-8257-440a-8a06-94bfc8e88816 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.573390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fa7f-8257-440a-8a06-94bfc8e88816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.573660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fa7f-8257-440a-8a06-94bfc8e88816 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.574273 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7fa7f-8257-440a-8a06-94bfc8e88816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.576805 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7fa7f-8257-440a-8a06-94bfc8e88816 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.597098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1408/2822] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:57.601176 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3e4d5-37c6-4933-ab7c-626aa2f31e72 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.611717 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3e4d5-37c6-4933-ab7c-626aa2f31e72 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.616263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f0727-e0ce-4538-a0e9-c9dba767d5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.616880 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f0727-e0ce-4538-a0e9-c9dba767d5cb admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.617159 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f0727-e0ce-4538-a0e9-c9dba767d5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.617428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f0727-e0ce-4538-a0e9-c9dba767d5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.618057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f0727-e0ce-4538-a0e9-c9dba767d5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.619672 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f0727-e0ce-4538-a0e9-c9dba767d5cb admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.629921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1415/2823] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:57.633808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fa7f-8257-440a-8a06-94bfc8e88816 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.643644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fa7f-8257-440a-8a06-94bfc8e88816 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.648249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3c6c-e17e-4932-98de-d1a3ad37af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.648835 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3c6c-e17e-4932-98de-d1a3ad37af0f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.649113 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3c6c-e17e-4932-98de-d1a3ad37af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.649380 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3c6c-e17e-4932-98de-d1a3ad37af0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.650005 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3c6c-e17e-4932-98de-d1a3ad37af0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.651631 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3c6c-e17e-4932-98de-d1a3ad37af0f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.662408 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1409/2824] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:57.666331 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f0727-e0ce-4538-a0e9-c9dba767d5cb admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.677081 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f0727-e0ce-4538-a0e9-c9dba767d5cb admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.681633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6b3f3-30c2-4edd-b650-ed150955a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.682273 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b3f3-30c2-4edd-b650-ed150955a4d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.682550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b3f3-30c2-4edd-b650-ed150955a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.682820 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b3f3-30c2-4edd-b650-ed150955a4d6 admin admin] PATH_INFO: `/v3/projects/e22ac0cf058c4350ba9e4f95d1cb0e44/users/2def580b18b7493d9b742bc937a350de/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.697498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b3f3-30c2-4edd-b650-ed150955a4d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e22ac0cf058c4350ba9e4f95d1cb0e44, user_id=2def580b18b7493d9b742bc937a350de, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.700182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b3f3-30c2-4edd-b650-ed150955a4d6 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.712628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1416/2825] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:57 2022] PUT /identity/v3/projects/e22ac0cf058c4350ba9e4f95d1cb0e44/users/2def580b18b7493d9b742bc937a350de/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:57.722715 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ebe5c-bb73-4a1c-ba49-148997dc7946 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.723029 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ebe5c-bb73-4a1c-ba49-148997dc7946 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.723461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ebe5c-bb73-4a1c-ba49-148997dc7946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.753377 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3e7ebe5c-bb73-4a1c-ba49-148997dc7946 None None] MFA Rules not processed for user `2def580b18b7493d9b742bc937a350de`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:57.754603 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3e7ebe5c-bb73-4a1c-ba49-148997dc7946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.762818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:57.821885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1410/2826] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:57.826490 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6b3f3-30c2-4edd-b650-ed150955a4d6 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.889312 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6b3f3-30c2-4edd-b650-ed150955a4d6 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.894574 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edac3bb-9de8-4300-8d71-3ff64ffe1189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.895214 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edac3bb-9de8-4300-8d71-3ff64ffe1189 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.895494 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edac3bb-9de8-4300-8d71-3ff64ffe1189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.895765 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edac3bb-9de8-4300-8d71-3ff64ffe1189 admin admin] PATH_INFO: `/v3/projects` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.896522 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edac3bb-9de8-4300-8d71-3ff64ffe1189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.898410 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edac3bb-9de8-4300-8d71-3ff64ffe1189 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.908285 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:57.934064 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1417/2827] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:57.938872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ebe5c-bb73-4a1c-ba49-148997dc7946 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.949644 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ebe5c-bb73-4a1c-ba49-148997dc7946 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.008513 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d94f73-38e8-4863-89a8-460e48f20d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.009125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94f73-38e8-4863-89a8-460e48f20d55 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.009461 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94f73-38e8-4863-89a8-460e48f20d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.009753 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94f73-38e8-4863-89a8-460e48f20d55 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.010282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d94f73-38e8-4863-89a8-460e48f20d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.012348 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d94f73-38e8-4863-89a8-460e48f20d55 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.035816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1411/2828] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:07:58.039759 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edac3bb-9de8-4300-8d71-3ff64ffe1189 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.052228 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edac3bb-9de8-4300-8d71-3ff64ffe1189 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.056949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9ab92-9cea-4573-a719-d00e70687209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.057550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ab92-9cea-4573-a719-d00e70687209 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.057835 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ab92-9cea-4573-a719-d00e70687209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.058116 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ab92-9cea-4573-a719-d00e70687209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.058787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9ab92-9cea-4573-a719-d00e70687209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.060437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9ab92-9cea-4573-a719-d00e70687209 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.070524 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1418/2829] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:07:58.074695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d94f73-38e8-4863-89a8-460e48f20d55 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.084707 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d94f73-38e8-4863-89a8-460e48f20d55 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.089298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a333d-3c03-4c60-b700-d4c51aa30432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.089936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a333d-3c03-4c60-b700-d4c51aa30432 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.090210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a333d-3c03-4c60-b700-d4c51aa30432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.091040 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a333d-3c03-4c60-b700-d4c51aa30432 admin admin] PATH_INFO: `/v3/projects/0083991bbcfe49c89a5a28b898dcc1bd/users/1408095111264543aee3690b4a479c86/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.105716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a333d-3c03-4c60-b700-d4c51aa30432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0083991bbcfe49c89a5a28b898dcc1bd, user_id=1408095111264543aee3690b4a479c86, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.109302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a333d-3c03-4c60-b700-d4c51aa30432 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.125962 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1412/2830] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:58 2022] PUT /identity/v3/projects/0083991bbcfe49c89a5a28b898dcc1bd/users/1408095111264543aee3690b4a479c86/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:58.127570 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9ab92-9cea-4573-a719-d00e70687209 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.138566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9ab92-9cea-4573-a719-d00e70687209 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.198858 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49056a-d297-4c80-b1f2-2ac84030dcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.199718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49056a-d297-4c80-b1f2-2ac84030dcba admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.200055 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49056a-d297-4c80-b1f2-2ac84030dcba admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.200582 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49056a-d297-4c80-b1f2-2ac84030dcba admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.201377 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49056a-d297-4c80-b1f2-2ac84030dcba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.203031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49056a-d297-4c80-b1f2-2ac84030dcba admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.215079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1419/2831] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:07:58.219182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a333d-3c03-4c60-b700-d4c51aa30432 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.230868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a333d-3c03-4c60-b700-d4c51aa30432 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.238571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.239290 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.239567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.239837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] PATH_INFO: `/v3/projects/0083991bbcfe49c89a5a28b898dcc1bd/users/1408095111264543aee3690b4a479c86/roles/2852c997f1e640a7afa94b5f113b12d6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.254064 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0083991bbcfe49c89a5a28b898dcc1bd, user_id=1408095111264543aee3690b4a479c86, role_id=2852c997f1e640a7afa94b5f113b12d6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.257335 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.269203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1413/2832] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:58 2022] PUT /identity/v3/projects/0083991bbcfe49c89a5a28b898dcc1bd/users/1408095111264543aee3690b4a479c86/roles/2852c997f1e640a7afa94b5f113b12d6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:58.280787 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a012f0-159f-4490-8e16-56a40b8cff57 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.281294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a012f0-159f-4490-8e16-56a40b8cff57 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.281635 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a012f0-159f-4490-8e16-56a40b8cff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.314335 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-51a012f0-159f-4490-8e16-56a40b8cff57 None None] MFA Rules not processed for user `1408095111264543aee3690b4a479c86`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:58.315928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-51a012f0-159f-4490-8e16-56a40b8cff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.391548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1420/2833] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:07:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:58.403812 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.406050 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.464437 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a4c0c-fef3-46c8-b0d6-1661fb538fd3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.471242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.471854 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.472131 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.472399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.482620 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.483093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a012f0-159f-4490-8e16-56a40b8cff57 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.484059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.484817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-51a012f0-159f-4490-8e16-56a40b8cff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.493772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1414/2834] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:58.539539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.540888 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.548908 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:58.551989 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:58.553282 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86309871-ea3a-43b8-8f5e-3071183a918b service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.560133 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87924b-d561-46f4-a639-0b86b08efa92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.560809 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87924b-d561-46f4-a639-0b86b08efa92 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.561156 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87924b-d561-46f4-a639-0b86b08efa92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.561489 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87924b-d561-46f4-a639-0b86b08efa92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.581955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a012f0-159f-4490-8e16-56a40b8cff57 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.587181 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.588149 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.588441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.588708 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.591408 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.602688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.604283 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.614498 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1421/2835] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:07:58.619654 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87924b-d561-46f4-a639-0b86b08efa92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.621067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87924b-d561-46f4-a639-0b86b08efa92 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.630143 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1415/2836] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:01.943751 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:02.005958 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94e411-93e7-4bda-b815-02c996442e14 service placement] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:02.010626 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:02.011278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:02.011559 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:02.011830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] PATH_INFO: `/v3/users/2def580b18b7493d9b742bc937a350de` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:02.012969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2def580b18b7493d9b742bc937a350de)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:02.014882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:02.060244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2def580b18b7493d9b742bc937a350de'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.067085 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2def580b18b7493d9b742bc937a350de'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.067772 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2def580b18b7493d9b742bc937a350de'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.068370 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2def580b18b7493d9b742bc937a350de'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.075882 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2def580b18b7493d9b742bc937a350de'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.093129 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1422/2837] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:01 2022] DELETE /identity/v3/users/2def580b18b7493d9b742bc937a350de => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:02.097739 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf3ce4-1554-4cac-ba0a-5e0e9471f92e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:02.098201 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf3ce4-1554-4cac-ba0a-5e0e9471f92e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:02.098325 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf3ce4-1554-4cac-ba0a-5e0e9471f92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:02.117009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-52cf3ce4-1554-4cac-ba0a-5e0e9471f92e None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:02.118212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-52cf3ce4-1554-4cac-ba0a-5e0e9471f92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:02.177587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1416/2838] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:02.186822 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:02.188252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:02.241120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:02.245422 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2732e-64d4-46b7-aeee-af76df56e65a admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:02.249711 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9fed5-348e-4c87-99f7-89bcee54eb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:02.250452 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9fed5-348e-4c87-99f7-89bcee54eb1d service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:02.250783 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9fed5-348e-4c87-99f7-89bcee54eb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:02.251758 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9fed5-348e-4c87-99f7-89bcee54eb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:02.260994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9fed5-348e-4c87-99f7-89bcee54eb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:02.262970 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9fed5-348e-4c87-99f7-89bcee54eb1d service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:02.272493 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1423/2839] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:02.530539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf3ce4-1554-4cac-ba0a-5e0e9471f92e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:02.531979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-52cf3ce4-1554-4cac-ba0a-5e0e9471f92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:02.543813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf3ce4-1554-4cac-ba0a-5e0e9471f92e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:02.548722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:02.549383 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:02.549661 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:02.549936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] PATH_INFO: `/v3/projects/e22ac0cf058c4350ba9e4f95d1cb0e44` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:02.551098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e22ac0cf058c4350ba9e4f95d1cb0e44)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:02.552980 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:02.599216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e22ac0cf058c4350ba9e4f95d1cb0e44', 'request_id': 'req-b073f951-e120-4349-9fb9-d5eae8fff8e6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.606185 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e22ac0cf058c4350ba9e4f95d1cb0e44', 'request_id': 'req-b073f951-e120-4349-9fb9-d5eae8fff8e6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.612572 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e22ac0cf058c4350ba9e4f95d1cb0e44', 'request_id': 'req-b073f951-e120-4349-9fb9-d5eae8fff8e6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.623456 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] The token cache is being invalidate because project e22ac0cf058c4350ba9e4f95d1cb0e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:02.623742 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.624415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.625950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1417/2840] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:02 2022] DELETE /identity/v3/projects/e22ac0cf058c4350ba9e4f95d1cb0e44 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:02.629897 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9fed5-348e-4c87-99f7-89bcee54eb1d service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:02.640022 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9fed5-348e-4c87-99f7-89bcee54eb1d service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:02.693833 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:02.694447 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:02.694724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:02.695028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] PATH_INFO: `/v3/users/1408095111264543aee3690b4a479c86` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:02.696155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1408095111264543aee3690b4a479c86)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:02.698048 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:02.742350 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1408095111264543aee3690b4a479c86'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.749785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1408095111264543aee3690b4a479c86'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.750428 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1408095111264543aee3690b4a479c86'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.751086 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1408095111264543aee3690b4a479c86'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.758441 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-213c8712-5671-48af-91a8-e79438002cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1408095111264543aee3690b4a479c86'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:02.776158 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1424/2841] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:02 2022] DELETE /identity/v3/users/1408095111264543aee3690b4a479c86 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:03.044454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.045874 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.061274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073f951-e120-4349-9fb9-d5eae8fff8e6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.086270 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:03.119653 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.120303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.120539 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.120917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] PATH_INFO: `/v3/projects/0083991bbcfe49c89a5a28b898dcc1bd` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.122645 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0083991bbcfe49c89a5a28b898dcc1bd)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.124558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.169761 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0083991bbcfe49c89a5a28b898dcc1bd', 'request_id': 'req-1296ac4b-be02-4efd-b7d4-1f893cd90018'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.176976 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0083991bbcfe49c89a5a28b898dcc1bd', 'request_id': 'req-1296ac4b-be02-4efd-b7d4-1f893cd90018'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.183171 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0083991bbcfe49c89a5a28b898dcc1bd', 'request_id': 'req-1296ac4b-be02-4efd-b7d4-1f893cd90018'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.193867 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] The token cache is being invalidate because project 0083991bbcfe49c89a5a28b898dcc1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:03.194150 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.194772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1296ac4b-be02-4efd-b7d4-1f893cd90018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.196447 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1418/2842] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:03 2022] DELETE /identity/v3/projects/0083991bbcfe49c89a5a28b898dcc1bd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:03.201089 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea473e4-7b2c-44dc-b28d-425a6ebbeac9 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.201555 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea473e4-7b2c-44dc-b28d-425a6ebbeac9 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.201655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea473e4-7b2c-44dc-b28d-425a6ebbeac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.221454 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-1ea473e4-7b2c-44dc-b28d-425a6ebbeac9 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:03.222644 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1ea473e4-7b2c-44dc-b28d-425a6ebbeac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.281856 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1425/2843] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:03.289873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cb64-5ac4-4b5b-a8f1-7e0f2af33c90 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.290092 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cb64-5ac4-4b5b-a8f1-7e0f2af33c90 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.290359 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cb64-5ac4-4b5b-a8f1-7e0f2af33c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.311043 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0d05cb64-5ac4-4b5b-a8f1-7e0f2af33c90 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:03.312243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0d05cb64-5ac4-4b5b-a8f1-7e0f2af33c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.320107 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1419/2844] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:03.324274 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea473e4-7b2c-44dc-b28d-425a6ebbeac9 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.334248 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea473e4-7b2c-44dc-b28d-425a6ebbeac9 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.338738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.339362 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.339638 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.339905 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.340540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:08:03.340837 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.342434 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.348919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:03.356809 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1426/2845] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:08:03.358574 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05cb64-5ac4-4b5b-a8f1-7e0f2af33c90 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.367784 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05cb64-5ac4-4b5b-a8f1-7e0f2af33c90 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.372913 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e7763-8bc4-43a6-a2fe-3a4b64bb6651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.373498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e7763-8bc4-43a6-a2fe-3a4b64bb6651 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.373778 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e7763-8bc4-43a6-a2fe-3a4b64bb6651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.374046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e7763-8bc4-43a6-a2fe-3a4b64bb6651 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.374667 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e7763-8bc4-43a6-a2fe-3a4b64bb6651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.376588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e7763-8bc4-43a6-a2fe-3a4b64bb6651 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.403285 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1420/2846] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:03 2022] POST /identity/v3/projects => generated 361 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:03.407648 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.417554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046d26-6129-4b25-bab5-7d67c1d26109 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.473277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d1a86-8453-4d63-94f9-038adb26d0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.473878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1a86-8453-4d63-94f9-038adb26d0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.474163 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1a86-8453-4d63-94f9-038adb26d0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.474437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1a86-8453-4d63-94f9-038adb26d0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.475106 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d1a86-8453-4d63-94f9-038adb26d0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.477016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d1a86-8453-4d63-94f9-038adb26d0a8 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.498071 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1427/2847] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:03 2022] POST /identity/v3/users => generated 281 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:03.502268 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e7763-8bc4-43a6-a2fe-3a4b64bb6651 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.513035 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e7763-8bc4-43a6-a2fe-3a4b64bb6651 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.517671 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aa0a9-da3c-4a9d-ae55-17d45ec8b360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.518262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aa0a9-da3c-4a9d-ae55-17d45ec8b360 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.518544 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aa0a9-da3c-4a9d-ae55-17d45ec8b360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.518813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aa0a9-da3c-4a9d-ae55-17d45ec8b360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.519506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134aa0a9-da3c-4a9d-ae55-17d45ec8b360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.521076 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134aa0a9-da3c-4a9d-ae55-17d45ec8b360 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.530731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1421/2848] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:03.534436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d1a86-8453-4d63-94f9-038adb26d0a8 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.545090 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d1a86-8453-4d63-94f9-038adb26d0a8 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.549396 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b265a3-9e48-4594-b421-8678d09d6eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.549999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b265a3-9e48-4594-b421-8678d09d6eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.550281 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b265a3-9e48-4594-b421-8678d09d6eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.550554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b265a3-9e48-4594-b421-8678d09d6eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.551233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b265a3-9e48-4594-b421-8678d09d6eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.552817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b265a3-9e48-4594-b421-8678d09d6eb4 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.563871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1428/2849] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:03.568104 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aa0a9-da3c-4a9d-ae55-17d45ec8b360 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.578911 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aa0a9-da3c-4a9d-ae55-17d45ec8b360 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.583542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.584182 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.584460 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.584731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] PATH_INFO: `/v3/projects/0ba1290b40b64ac79575ef693ad39dd8/users/127f0e3c2b5a4304b43ce9376cbc4707/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.599191 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba1290b40b64ac79575ef693ad39dd8, user_id=127f0e3c2b5a4304b43ce9376cbc4707, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.601808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.615116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1422/2850] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:03 2022] PUT /identity/v3/projects/0ba1290b40b64ac79575ef693ad39dd8/users/127f0e3c2b5a4304b43ce9376cbc4707/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:03.625939 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045f791-a2fa-4c43-b7b1-f999ce588f4b None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.626226 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045f791-a2fa-4c43-b7b1-f999ce588f4b None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.626499 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045f791-a2fa-4c43-b7b1-f999ce588f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.653028 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-3045f791-a2fa-4c43-b7b1-f999ce588f4b None None] MFA Rules not processed for user `127f0e3c2b5a4304b43ce9376cbc4707`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:03.654210 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-3045f791-a2fa-4c43-b7b1-f999ce588f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.719867 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1429/2851] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:03.756126 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.758028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.815316 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f009608-fc2c-45a4-893a-a9b89cde48db admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.818813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83fea3-c9f5-488a-9356-ea75a8b5a774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.819450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83fea3-c9f5-488a-9356-ea75a8b5a774 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.819712 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83fea3-c9f5-488a-9356-ea75a8b5a774 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.819983 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83fea3-c9f5-488a-9356-ea75a8b5a774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.826523 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:03.828772 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83fea3-c9f5-488a-9356-ea75a8b5a774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.830537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83fea3-c9f5-488a-9356-ea75a8b5a774 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.841638 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1423/2852] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:03.857176 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045f791-a2fa-4c43-b7b1-f999ce588f4b None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.918155 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045f791-a2fa-4c43-b7b1-f999ce588f4b None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.923053 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.923677 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.923961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.924237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] PATH_INFO: `/v3/users/127f0e3c2b5a4304b43ce9376cbc4707` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.925355 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127f0e3c2b5a4304b43ce9376cbc4707)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.927296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.932556 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:03.972646 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127f0e3c2b5a4304b43ce9376cbc4707'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.979308 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127f0e3c2b5a4304b43ce9376cbc4707'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.979949 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127f0e3c2b5a4304b43ce9376cbc4707'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.980542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127f0e3c2b5a4304b43ce9376cbc4707'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:03.987310 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127f0e3c2b5a4304b43ce9376cbc4707'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:04.004587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1430/2853] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:03 2022] DELETE /identity/v3/users/127f0e3c2b5a4304b43ce9376cbc4707 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:04.009085 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df5d7-40a9-4ee8-9dea-8b00413294b8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.009375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df5d7-40a9-4ee8-9dea-8b00413294b8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.009649 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1df5d7-40a9-4ee8-9dea-8b00413294b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.028923 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bb1df5d7-40a9-4ee8-9dea-8b00413294b8 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:04.030120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bb1df5d7-40a9-4ee8-9dea-8b00413294b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:04.087507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1424/2854] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:04.097167 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.098540 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:04.154674 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ac229-81c0-45b4-8d47-cb08b31b80c9 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.158286 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32faaeef-16f9-46b3-a24a-9a43fed2baad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.158877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faaeef-16f9-46b3-a24a-9a43fed2baad service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.159184 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faaeef-16f9-46b3-a24a-9a43fed2baad service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.159451 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faaeef-16f9-46b3-a24a-9a43fed2baad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.168067 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faaeef-16f9-46b3-a24a-9a43fed2baad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.169469 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faaeef-16f9-46b3-a24a-9a43fed2baad service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.178874 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1431/2855] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:04.440123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1df5d7-40a9-4ee8-9dea-8b00413294b8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.441547 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bb1df5d7-40a9-4ee8-9dea-8b00413294b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:04.455450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1df5d7-40a9-4ee8-9dea-8b00413294b8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.461571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.462442 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.462724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.463083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] PATH_INFO: `/v3/projects/0ba1290b40b64ac79575ef693ad39dd8` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.464321 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba1290b40b64ac79575ef693ad39dd8)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.466257 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.514168 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba1290b40b64ac79575ef693ad39dd8', 'request_id': 'req-1f1e4f32-cc57-4f14-a9bd-29121869e659'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:04.521197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba1290b40b64ac79575ef693ad39dd8', 'request_id': 'req-1f1e4f32-cc57-4f14-a9bd-29121869e659'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:04.527690 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba1290b40b64ac79575ef693ad39dd8', 'request_id': 'req-1f1e4f32-cc57-4f14-a9bd-29121869e659'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:04.537883 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] The token cache is being invalidate because project 0ba1290b40b64ac79575ef693ad39dd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:04.538166 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:04.538781 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f1e4f32-cc57-4f14-a9bd-29121869e659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:04.540299 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1425/2856] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:04 2022] DELETE /identity/v3/projects/0ba1290b40b64ac79575ef693ad39dd8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:04.545679 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223a80f-cbba-4880-80bd-180737cd9072 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.546429 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223a80f-cbba-4880-80bd-180737cd9072 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.546539 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223a80f-cbba-4880-80bd-180737cd9072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.567154 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-7223a80f-cbba-4880-80bd-180737cd9072 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:04.568888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7223a80f-cbba-4880-80bd-180737cd9072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:04.626571 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1432/2857] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:04.635477 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06645f6-a15e-4536-872c-84993e28ee7a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.635750 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06645f6-a15e-4536-872c-84993e28ee7a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.636030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06645f6-a15e-4536-872c-84993e28ee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.655623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b06645f6-a15e-4536-872c-84993e28ee7a None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:04.656842 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b06645f6-a15e-4536-872c-84993e28ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:04.665007 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1426/2858] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:04.669760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223a80f-cbba-4880-80bd-180737cd9072 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.679808 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223a80f-cbba-4880-80bd-180737cd9072 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.685001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.685596 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.685875 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.686146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.686785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:08:04.687125 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.688721 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.699999 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1433/2859] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:08:04.704111 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06645f6-a15e-4536-872c-84993e28ee7a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.713228 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06645f6-a15e-4536-872c-84993e28ee7a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.718637 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f8554-1a42-4098-b3d4-ee55382c1263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.719276 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f8554-1a42-4098-b3d4-ee55382c1263 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.719585 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f8554-1a42-4098-b3d4-ee55382c1263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.719863 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f8554-1a42-4098-b3d4-ee55382c1263 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.720525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f8554-1a42-4098-b3d4-ee55382c1263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.722405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f8554-1a42-4098-b3d4-ee55382c1263 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.750369 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1427/2860] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:04 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:04.754415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.764481 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9cfe1-6425-4ce2-abbd-7bdbf9d61584 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.776246 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:04.822249 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac627ca6-9935-4306-9b96-fba4d031845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.822848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627ca6-9935-4306-9b96-fba4d031845f admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.823164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627ca6-9935-4306-9b96-fba4d031845f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.823435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627ca6-9935-4306-9b96-fba4d031845f admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.824051 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac627ca6-9935-4306-9b96-fba4d031845f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.825955 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac627ca6-9935-4306-9b96-fba4d031845f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.847122 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1434/2861] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:04 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:04.851262 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f8554-1a42-4098-b3d4-ee55382c1263 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.862116 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f8554-1a42-4098-b3d4-ee55382c1263 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.866905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb462744-0252-480f-b57b-98854bb11d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.867507 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462744-0252-480f-b57b-98854bb11d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.867782 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462744-0252-480f-b57b-98854bb11d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.868154 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462744-0252-480f-b57b-98854bb11d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.868791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb462744-0252-480f-b57b-98854bb11d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.870385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb462744-0252-480f-b57b-98854bb11d07 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.880974 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1428/2862] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:08:04.884755 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac627ca6-9935-4306-9b96-fba4d031845f admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.895643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac627ca6-9935-4306-9b96-fba4d031845f admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.900253 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3103a8c-def9-4d46-ab65-f211b5302af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.900842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3103a8c-def9-4d46-ab65-f211b5302af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.901120 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3103a8c-def9-4d46-ab65-f211b5302af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.901388 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3103a8c-def9-4d46-ab65-f211b5302af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.902017 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3103a8c-def9-4d46-ab65-f211b5302af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.904108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3103a8c-def9-4d46-ab65-f211b5302af5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.919844 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1435/2863] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:04.924075 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb462744-0252-480f-b57b-98854bb11d07 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.934141 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb462744-0252-480f-b57b-98854bb11d07 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.938836 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.939538 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.939816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.940088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] PATH_INFO: `/v3/projects/38aeafae184a4a35859c9e11d25e2aff/users/73c0a56653d8404f87b760decb8c2f89/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.954483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38aeafae184a4a35859c9e11d25e2aff, user_id=73c0a56653d8404f87b760decb8c2f89, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.957281 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.970315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1429/2864] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:04 2022] PUT /identity/v3/projects/38aeafae184a4a35859c9e11d25e2aff/users/73c0a56653d8404f87b760decb8c2f89/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:04.975587 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb24c-0c94-409d-8ac6-9e7200e85ca8 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.975879 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb24c-0c94-409d-8ac6-9e7200e85ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.976179 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb24c-0c94-409d-8ac6-9e7200e85ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.995323 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-adbeb24c-0c94-409d-8ac6-9e7200e85ca8 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:04.996554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-adbeb24c-0c94-409d-8ac6-9e7200e85ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:05.054779 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1436/2865] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:05.064065 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:05.065808 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:05.084868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:05.125165 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035ffe2-99e5-486e-88ac-e3029b8d1080 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:05.128793 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:05.129382 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:05.129658 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:05.129927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:05.139287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:05.140633 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:05.149927 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1430/2866] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:05.972516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6fce-d358-4454-9847-415007203a90 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:05.973020 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6fce-d358-4454-9847-415007203a90 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:05.973340 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6fce-d358-4454-9847-415007203a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:06.003236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-d86e6fce-d358-4454-9847-415007203a90 None None] MFA Rules not processed for user `73c0a56653d8404f87b760decb8c2f89`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:06.004591 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d86e6fce-d358-4454-9847-415007203a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:06.078934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1437/2867] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:06.089542 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:06.091067 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:06.152497 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786e35d-17c3-4324-a9f6-38f2c805568e service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:06.156385 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:06.157034 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:06.157907 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:06.158188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:06.170410 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:06.172083 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:06.181741 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1431/2868] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:07.558444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5cd8f-4ce9-4b37-8a70-d5786bcf2f40 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:07.558953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5cd8f-4ce9-4b37-8a70-d5786bcf2f40 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:07.558953 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5cd8f-4ce9-4b37-8a70-d5786bcf2f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:07.579079 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-9fd5cd8f-4ce9-4b37-8a70-d5786bcf2f40 None None] MFA Rules not processed for user `73c0a56653d8404f87b760decb8c2f89`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:07.580257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9fd5cd8f-4ce9-4b37-8a70-d5786bcf2f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:07.586956 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1438/2869] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:16.177684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=45 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:17.573817 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:24.362363 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:24.363957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:24.382051 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce13a-77d9-460b-8d3d-f13775f799de service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:24.387420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:24.388125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:24.391888 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:24.392194 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] PATH_INFO: `/v3/users/0243f437de8b439984375303ff49f9e3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:24.393450 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0243f437de8b439984375303ff49f9e3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:24.395464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:24.468211 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0243f437de8b439984375303ff49f9e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.476220 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0243f437de8b439984375303ff49f9e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.477298 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0243f437de8b439984375303ff49f9e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.478019 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0243f437de8b439984375303ff49f9e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.488821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-14791343-2f7a-47ee-b7aa-53fb49625785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0243f437de8b439984375303ff49f9e3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.510816 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1432/2870] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:24 2022] DELETE /identity/v3/users/0243f437de8b439984375303ff49f9e3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:24.704391 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5cd8f-4ce9-4b37-8a70-d5786bcf2f40 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:24.715922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5cd8f-4ce9-4b37-8a70-d5786bcf2f40 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:24.775381 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:24.776015 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:24.776294 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:24.776564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] PATH_INFO: `/v3/projects/0223b57233664d0785c96ec98335e307` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:24.777722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0223b57233664d0785c96ec98335e307)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:24.779795 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:24.826738 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0223b57233664d0785c96ec98335e307', 'request_id': 'req-a04bad0f-9595-453a-9add-bcc619d26675'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.834264 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0223b57233664d0785c96ec98335e307', 'request_id': 'req-a04bad0f-9595-453a-9add-bcc619d26675'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.840678 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0223b57233664d0785c96ec98335e307', 'request_id': 'req-a04bad0f-9595-453a-9add-bcc619d26675'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.850805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] The token cache is being invalidate because project 0223b57233664d0785c96ec98335e307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:24.851119 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.851740 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a04bad0f-9595-453a-9add-bcc619d26675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:24.853257 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1439/2871] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:24 2022] DELETE /identity/v3/projects/0223b57233664d0785c96ec98335e307 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:25.743234 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136600-1e17-4daf-86cc-859110c7a894 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:25.743813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136600-1e17-4daf-86cc-859110c7a894 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:25.743813 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136600-1e17-4daf-86cc-859110c7a894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:25.762563 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-22136600-1e17-4daf-86cc-859110c7a894 None None] MFA Rules not processed for user `73c0a56653d8404f87b760decb8c2f89`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:25.763831 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-22136600-1e17-4daf-86cc-859110c7a894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:25.817597 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1433/2872] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:08:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:34.847889 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:35.811030 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:47.034996 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2148dd-9911-4b6c-8040-ac2c85fba6d8 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:47.035542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2148dd-9911-4b6c-8040-ac2c85fba6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:47.035542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2148dd-9911-4b6c-8040-ac2c85fba6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:47.065209 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bb2148dd-9911-4b6c-8040-ac2c85fba6d8 None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:47.066415 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bb2148dd-9911-4b6c-8040-ac2c85fba6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:47.122730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1440/2873] 10.222.0.52 () {58 vars in 1009 bytes} [Mon Oct 10 12:08:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:47.168999 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22136600-1e17-4daf-86cc-859110c7a894 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:47.170413 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-22136600-1e17-4daf-86cc-859110c7a894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:47.242797 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22136600-1e17-4daf-86cc-859110c7a894 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:47.248253 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6fc7d-f4d2-4c2c-9245-62385e2203fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:47.248859 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fc7d-f4d2-4c2c-9245-62385e2203fc service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:47.249145 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fc7d-f4d2-4c2c-9245-62385e2203fc service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:47.249423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fc7d-f4d2-4c2c-9245-62385e2203fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:47.258028 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6fc7d-f4d2-4c2c-9245-62385e2203fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:47.259675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6fc7d-f4d2-4c2c-9245-62385e2203fc service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:47.268233 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1434/2874] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:08:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:08:57.116704 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=29 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:57.264502 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=32 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:03.595033 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c9a11-d41a-4aff-8698-d54633d472ca None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:03.595468 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c9a11-d41a-4aff-8698-d54633d472ca None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:03.595581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c9a11-d41a-4aff-8698-d54633d472ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:03.614394 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-6c1c9a11-d41a-4aff-8698-d54633d472ca None None] MFA Rules not processed for user `cf591850b9914f2f9089c8b6df730b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:03.615643 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6c1c9a11-d41a-4aff-8698-d54633d472ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:03.622624 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1441/2875] 10.222.0.52 () {58 vars in 967 bytes} [Mon Oct 10 12:09:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:03.634804 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6fc7d-f4d2-4c2c-9245-62385e2203fc service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:03.645851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6fc7d-f4d2-4c2c-9245-62385e2203fc service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:03.650693 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d82ba2-1892-44a4-896c-cada76c2df40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91df91b1ea4949119245c86bf52e32f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:03.651302 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d82ba2-1892-44a4-896c-cada76c2df40 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:03.651580 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d82ba2-1892-44a4-896c-cada76c2df40 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:03.651845 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d82ba2-1892-44a4-896c-cada76c2df40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:03.660292 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d82ba2-1892-44a4-896c-cada76c2df40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:03.661837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d82ba2-1892-44a4-896c-cada76c2df40 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:03.671875 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1435/2876] 10.222.0.52 () {60 vars in 1384 bytes} [Mon Oct 10 12:09:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:09:05.571356 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c9a11-d41a-4aff-8698-d54633d472ca None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:05.573023 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-6c1c9a11-d41a-4aff-8698-d54633d472ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:05.639436 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c9a11-d41a-4aff-8698-d54633d472ca None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:05.644258 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:05.644888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:05.645168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:05.645440 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] PATH_INFO: `/v3/users/73c0a56653d8404f87b760decb8c2f89` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:05.646549 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c0a56653d8404f87b760decb8c2f89)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:05.648507 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:05.694098 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c0a56653d8404f87b760decb8c2f89'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:05.701344 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c0a56653d8404f87b760decb8c2f89'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:05.702025 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c0a56653d8404f87b760decb8c2f89'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:05.702680 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c0a56653d8404f87b760decb8c2f89'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:05.710420 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-a2ff4948-7cf9-4a37-9fd2-f87fd05322ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c0a56653d8404f87b760decb8c2f89'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:05.728521 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1442/2877] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:05 2022] DELETE /identity/v3/users/73c0a56653d8404f87b760decb8c2f89 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:05.896454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d82ba2-1892-44a4-896c-cada76c2df40 service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:05.910623 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d82ba2-1892-44a4-896c-cada76c2df40 service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:05.969094 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:05.969716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:05.969994 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:05.970265 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] PATH_INFO: `/v3/projects/38aeafae184a4a35859c9e11d25e2aff` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:05.971427 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=38aeafae184a4a35859c9e11d25e2aff)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:05.973314 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:06.019266 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38aeafae184a4a35859c9e11d25e2aff', 'request_id': 'req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:06.026208 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38aeafae184a4a35859c9e11d25e2aff', 'request_id': 'req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:06.033012 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38aeafae184a4a35859c9e11d25e2aff', 'request_id': 'req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:06.043142 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] The token cache is being invalidate because project 38aeafae184a4a35859c9e11d25e2aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:06.043431 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:06.044044 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-93fcc9df-ab9a-4993-af61-d6e12fc5ff2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:06.045828 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1436/2878] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:05 2022] DELETE /identity/v3/projects/38aeafae184a4a35859c9e11d25e2aff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:06.051026 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7cce3-0b53-4b9c-8e40-70862c025fdb None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.051321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7cce3-0b53-4b9c-8e40-70862c025fdb None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.051602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7cce3-0b53-4b9c-8e40-70862c025fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.071332 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-66b7cce3-0b53-4b9c-8e40-70862c025fdb None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:06.072561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-66b7cce3-0b53-4b9c-8e40-70862c025fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:06.129998 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1443/2879] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:06.137685 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cbf10-0d27-40a5-8997-a8cc8dd55ec7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.137973 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cbf10-0d27-40a5-8997-a8cc8dd55ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.138247 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cbf10-0d27-40a5-8997-a8cc8dd55ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.159274 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ee8cbf10-0d27-40a5-8997-a8cc8dd55ec7 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:06.160405 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ee8cbf10-0d27-40a5-8997-a8cc8dd55ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:06.168293 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1437/2880] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:06.173277 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7cce3-0b53-4b9c-8e40-70862c025fdb None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:06.182769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7cce3-0b53-4b9c-8e40-70862c025fdb None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:06.187361 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:06.187961 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.188243 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.188519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.189164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:06.189467 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:06.191092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:06.199602 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:06.202828 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1444/2881] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:06.207277 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cbf10-0d27-40a5-8997-a8cc8dd55ec7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:06.216386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cbf10-0d27-40a5-8997-a8cc8dd55ec7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:06.221998 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0077-e1fd-4d14-8ad0-51fb05d6ef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:06.222592 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0077-e1fd-4d14-8ad0-51fb05d6ef75 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.222872 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0077-e1fd-4d14-8ad0-51fb05d6ef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.223222 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0077-e1fd-4d14-8ad0-51fb05d6ef75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.223851 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0077-e1fd-4d14-8ad0-51fb05d6ef75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:06.225722 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0077-e1fd-4d14-8ad0-51fb05d6ef75 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:06.262453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1438/2882] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:06 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:06.266531 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:06.276435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961c5cd-d8c2-410e-b4c4-2c77348201d5 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:06.332296 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a450980-d600-4048-abb5-6e585731724b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:06.332908 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a450980-d600-4048-abb5-6e585731724b admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.333182 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a450980-d600-4048-abb5-6e585731724b admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.333482 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a450980-d600-4048-abb5-6e585731724b admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.334141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a450980-d600-4048-abb5-6e585731724b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:06.336080 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a450980-d600-4048-abb5-6e585731724b admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:06.357170 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1445/2883] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:06 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:06.361896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0077-e1fd-4d14-8ad0-51fb05d6ef75 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:06.372008 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0077-e1fd-4d14-8ad0-51fb05d6ef75 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:06.377435 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318cea-b9a6-4662-ba74-b84c2cadb42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:06.378036 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318cea-b9a6-4662-ba74-b84c2cadb42e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.378315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318cea-b9a6-4662-ba74-b84c2cadb42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.378587 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318cea-b9a6-4662-ba74-b84c2cadb42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.379283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318cea-b9a6-4662-ba74-b84c2cadb42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:06.381242 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318cea-b9a6-4662-ba74-b84c2cadb42e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:06.391700 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1439/2884] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:06.395444 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a450980-d600-4048-abb5-6e585731724b admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:06.405419 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a450980-d600-4048-abb5-6e585731724b admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:06.411979 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416d063-6580-4d8f-9248-b443050d7a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:06.412564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416d063-6580-4d8f-9248-b443050d7a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.412811 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416d063-6580-4d8f-9248-b443050d7a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.413077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416d063-6580-4d8f-9248-b443050d7a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.413725 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416d063-6580-4d8f-9248-b443050d7a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:06.415437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416d063-6580-4d8f-9248-b443050d7a28 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:06.427132 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1446/2885] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:06.431932 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318cea-b9a6-4662-ba74-b84c2cadb42e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:06.442430 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318cea-b9a6-4662-ba74-b84c2cadb42e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:06.447294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:06.447947 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.448229 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.448503 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] PATH_INFO: `/v3/projects/6008262991844b799ce658e5ed6e0855/users/7f37eaf6a83a4afb83eb5d84addae32e/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.464199 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6008262991844b799ce658e5ed6e0855, user_id=7f37eaf6a83a4afb83eb5d84addae32e, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:06.466830 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:06.480181 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1440/2886] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:06 2022] PUT /identity/v3/projects/6008262991844b799ce658e5ed6e0855/users/7f37eaf6a83a4afb83eb5d84addae32e/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:06.491675 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e421c-0ac9-44e6-9b20-a0bad46486fc None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.491960 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e421c-0ac9-44e6-9b20-a0bad46486fc None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.492232 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e421c-0ac9-44e6-9b20-a0bad46486fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.520945 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-bd8e421c-0ac9-44e6-9b20-a0bad46486fc None None] MFA Rules not processed for user `7f37eaf6a83a4afb83eb5d84addae32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:06.522149 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bd8e421c-0ac9-44e6-9b20-a0bad46486fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:06.586561 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1447/2887] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:09:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:06.598120 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:06.599823 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:06.606296 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:06.657724 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09cef-291b-4fd9-ac3b-6a5dbc982554 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:06.661311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d5ccc-a9f5-491f-b7f1-5a29c686f832 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6774fdd8b2e46b6a7d00919568f9fd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:06.661909 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d5ccc-a9f5-491f-b7f1-5a29c686f832 service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:06.662241 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d5ccc-a9f5-491f-b7f1-5a29c686f832 service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:06.662464 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d5ccc-a9f5-491f-b7f1-5a29c686f832 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:06.671014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d5ccc-a9f5-491f-b7f1-5a29c686f832 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:06.672917 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d5ccc-a9f5-491f-b7f1-5a29c686f832 service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:06.681526 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1441/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:09.044144 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e421c-0ac9-44e6-9b20-a0bad46486fc None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:09.046126 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-bd8e421c-0ac9-44e6-9b20-a0bad46486fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:09.107360 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e421c-0ac9-44e6-9b20-a0bad46486fc None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:09.110877 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:09.111510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:09.111924 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:09.112202 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:09.114078 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:09.137633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:09.206093 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:09.207659 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:09.231043 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1448/2889] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:10.417483 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d5ccc-a9f5-491f-b7f1-5a29c686f832 service glance] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:10.428420 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d5ccc-a9f5-491f-b7f1-5a29c686f832 service glance] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:10.432216 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:10.432834 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:10.433117 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:10.433399 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:10.435212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:10.449676 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:10.451121 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:10.462616 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1442/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:19.221693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=23 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:20.457537 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=30 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:21.396325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:21.405057 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e3bd9-e5ed-42ed-bd23-a359241b810f service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:21.408767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:21.409302 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:21.409578 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:21.409845 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:21.411576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:21.476021 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:21.477453 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:21.487349 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1449/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:24.566014 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:24.568047 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:24.651765 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cc6f9-d406-4da8-a4b5-0717fdc4ffb8 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:24.657886 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:24.659189 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:24.659568 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:24.659936 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:24.662512 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:24.676311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:24.678069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:24.698124 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1443/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:27.495797 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:27.497321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:27.510763 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc6d15-fde3-4ca4-823d-d7daed6e13ac service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:27.515722 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:27.516365 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:27.516654 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:27.516933 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] PATH_INFO: `/v3/users/ce6a1b5ca7584f90bfe83b73aff1335a` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:27.518728 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6a1b5ca7584f90bfe83b73aff1335a)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:27.520805 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:27.569936 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6a1b5ca7584f90bfe83b73aff1335a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.581606 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6a1b5ca7584f90bfe83b73aff1335a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.582321 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6a1b5ca7584f90bfe83b73aff1335a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.582940 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6a1b5ca7584f90bfe83b73aff1335a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.593691 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6a1b5ca7584f90bfe83b73aff1335a'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.611803 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1450/2893] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:27 2022] DELETE /identity/v3/users/ce6a1b5ca7584f90bfe83b73aff1335a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:27.791338 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:27.811796 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1927482-9dc2-4bf6-a68b-31556f222608 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:27.896982 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:27.897619 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:27.897900 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:27.898173 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] PATH_INFO: `/v3/projects/638435e7c9ea431aa6ad3674ee6ff2ae` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:27.899326 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=638435e7c9ea431aa6ad3674ee6ff2ae)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:27.901219 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:27.948849 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '638435e7c9ea431aa6ad3674ee6ff2ae', 'request_id': 'req-3bbce04d-7829-4b45-9271-6f8341209d82'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.955375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '638435e7c9ea431aa6ad3674ee6ff2ae', 'request_id': 'req-3bbce04d-7829-4b45-9271-6f8341209d82'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.961857 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '638435e7c9ea431aa6ad3674ee6ff2ae', 'request_id': 'req-3bbce04d-7829-4b45-9271-6f8341209d82'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.972197 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] The token cache is being invalidate because project 638435e7c9ea431aa6ad3674ee6ff2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:27.972423 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.973060 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.974545 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1444/2894] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:27 2022] DELETE /identity/v3/projects/638435e7c9ea431aa6ad3674ee6ff2ae => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:27.978525 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:27.980042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:27.993535 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d805dda-f7e2-4e76-bd23-48ce08dd4db9 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:28.047599 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:28.048203 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.048542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.049358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] PATH_INFO: `/v3/users/06d7f6b94e574f1a89c4c41b55d79044` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.050514 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d7f6b94e574f1a89c4c41b55d79044)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:28.052471 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:28.057718 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:28.098449 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d7f6b94e574f1a89c4c41b55d79044'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.105263 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d7f6b94e574f1a89c4c41b55d79044'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.105911 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d7f6b94e574f1a89c4c41b55d79044'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.106508 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d7f6b94e574f1a89c4c41b55d79044'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.114244 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d7f6b94e574f1a89c4c41b55d79044'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.131814 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1451/2895] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:27 2022] DELETE /identity/v3/users/06d7f6b94e574f1a89c4c41b55d79044 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:28.308214 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:28.309733 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:28.323716 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbce04d-7829-4b45-9271-6f8341209d82 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:28.340905 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:28.387230 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:28.387870 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.388175 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.388453 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] PATH_INFO: `/v3/projects/53dfa853fdbd4749a0521835994ce87b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.389579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53dfa853fdbd4749a0521835994ce87b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:28.391586 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:28.438837 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53dfa853fdbd4749a0521835994ce87b', 'request_id': 'req-55837a95-8309-4be2-ac95-bbe3be020359'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.446732 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53dfa853fdbd4749a0521835994ce87b', 'request_id': 'req-55837a95-8309-4be2-ac95-bbe3be020359'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.453350 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53dfa853fdbd4749a0521835994ce87b', 'request_id': 'req-55837a95-8309-4be2-ac95-bbe3be020359'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.463610 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] The token cache is being invalidate because project 53dfa853fdbd4749a0521835994ce87b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:28.463896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.464524 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55837a95-8309-4be2-ac95-bbe3be020359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.466000 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1445/2896] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:28 2022] DELETE /identity/v3/projects/53dfa853fdbd4749a0521835994ce87b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:38.008075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.009564 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.022164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e015e-9c83-4027-a714-9e05efe90c7e admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.076946 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.077566 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.077848 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.078121 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] PATH_INFO: `/v3/users/7f37eaf6a83a4afb83eb5d84addae32e` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.079259 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f37eaf6a83a4afb83eb5d84addae32e)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.081188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:38.126001 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f37eaf6a83a4afb83eb5d84addae32e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.132888 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f37eaf6a83a4afb83eb5d84addae32e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.133534 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f37eaf6a83a4afb83eb5d84addae32e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.134135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f37eaf6a83a4afb83eb5d84addae32e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.141072 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f37eaf6a83a4afb83eb5d84addae32e'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.158077 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1452/2897] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:38 2022] DELETE /identity/v3/users/7f37eaf6a83a4afb83eb5d84addae32e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:38.163488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0447da0-1fa4-462e-bc6c-a92e7f4c0918 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.163955 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0447da0-1fa4-462e-bc6c-a92e7f4c0918 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.164059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0447da0-1fa4-462e-bc6c-a92e7f4c0918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.183896 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f0447da0-1fa4-462e-bc6c-a92e7f4c0918 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:38.185096 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f0447da0-1fa4-462e-bc6c-a92e7f4c0918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.242204 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1446/2898] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:38.251838 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.253262 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.259760 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:38.311576 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043262a-b8ae-468c-8ccc-b902973790c5 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.315270 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be81013-93df-4797-8c1c-3fcf2e1e9bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.315854 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be81013-93df-4797-8c1c-3fcf2e1e9bed service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.316142 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be81013-93df-4797-8c1c-3fcf2e1e9bed service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.316421 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be81013-93df-4797-8c1c-3fcf2e1e9bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.325748 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be81013-93df-4797-8c1c-3fcf2e1e9bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.327236 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be81013-93df-4797-8c1c-3fcf2e1e9bed service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:38.337162 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1453/2899] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:38.598401 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0447da0-1fa4-462e-bc6c-a92e7f4c0918 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.600263 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f0447da0-1fa4-462e-bc6c-a92e7f4c0918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.615231 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0447da0-1fa4-462e-bc6c-a92e7f4c0918 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.620571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.621215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.621499 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.621790 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] PATH_INFO: `/v3/projects/6008262991844b799ce658e5ed6e0855` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.623101 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6008262991844b799ce658e5ed6e0855)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.625219 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:38.670336 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:38.673381 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6008262991844b799ce658e5ed6e0855', 'request_id': 'req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.681543 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6008262991844b799ce658e5ed6e0855', 'request_id': 'req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.688125 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6008262991844b799ce658e5ed6e0855', 'request_id': 'req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.698411 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] The token cache is being invalidate because project 6008262991844b799ce658e5ed6e0855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:38.698695 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.699353 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3256cb06-9dd6-4e4f-b72c-a0d3327a768f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.700819 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1447/2900] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:38 2022] DELETE /identity/v3/projects/6008262991844b799ce658e5ed6e0855 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:38.705688 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b1bab-381b-4a13-bb13-59e50e0ddc60 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.705975 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b1bab-381b-4a13-bb13-59e50e0ddc60 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.706252 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b1bab-381b-4a13-bb13-59e50e0ddc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.727104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-666b1bab-381b-4a13-bb13-59e50e0ddc60 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:38.728297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-666b1bab-381b-4a13-bb13-59e50e0ddc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.786161 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1454/2901] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:38.794099 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e4ead-516e-48b4-a517-a8e9ae7a7980 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.794386 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e4ead-516e-48b4-a517-a8e9ae7a7980 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.794949 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e4ead-516e-48b4-a517-a8e9ae7a7980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.814291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8d0e4ead-516e-48b4-a517-a8e9ae7a7980 None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:38.816046 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8d0e4ead-516e-48b4-a517-a8e9ae7a7980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.824071 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1448/2902] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:38.828114 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b1bab-381b-4a13-bb13-59e50e0ddc60 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.838325 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b1bab-381b-4a13-bb13-59e50e0ddc60 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.842804 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.843430 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.843715 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.843994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] PATH_INFO: `/v3/domains` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.844633 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81457) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:38.845075 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.846692 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:38.857818 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1455/2903] 10.222.0.52 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:38.861791 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e4ead-516e-48b4-a517-a8e9ae7a7980 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.871009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e4ead-516e-48b4-a517-a8e9ae7a7980 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.875950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e904-139d-4b88-86a8-de870711164b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.876534 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e904-139d-4b88-86a8-de870711164b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.876818 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e904-139d-4b88-86a8-de870711164b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.877088 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e904-139d-4b88-86a8-de870711164b admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.878303 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366e904-139d-4b88-86a8-de870711164b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.880215 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366e904-139d-4b88-86a8-de870711164b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:38.907406 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1449/2904] 10.222.0.52 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:38 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:38.911724 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.921919 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf30cd4-4b7d-44c6-b286-1b9d22637123 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.976501 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb7c11-6d73-4213-9eca-514db1680461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.977104 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb7c11-6d73-4213-9eca-514db1680461 admin admin] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.977387 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb7c11-6d73-4213-9eca-514db1680461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.977663 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb7c11-6d73-4213-9eca-514db1680461 admin admin] PATH_INFO: `/v3/users` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.978278 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb7c11-6d73-4213-9eca-514db1680461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.980195 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb7c11-6d73-4213-9eca-514db1680461 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:39.001237 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1456/2905] 10.222.0.52 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:38 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:39.005977 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e904-139d-4b88-86a8-de870711164b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:39.016210 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366e904-139d-4b88-86a8-de870711164b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:39.021278 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d831479-a6fa-4d47-b8c9-b86e161cc69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:39.021868 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d831479-a6fa-4d47-b8c9-b86e161cc69c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:39.022148 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d831479-a6fa-4d47-b8c9-b86e161cc69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:39.022418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d831479-a6fa-4d47-b8c9-b86e161cc69c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:39.023090 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d831479-a6fa-4d47-b8c9-b86e161cc69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:39.024660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d831479-a6fa-4d47-b8c9-b86e161cc69c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:39.035041 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1450/2906] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:39.039047 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb7c11-6d73-4213-9eca-514db1680461 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:39.049108 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb7c11-6d73-4213-9eca-514db1680461 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:39.053696 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a2926-96cc-492a-a9a4-b50754c6ce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:39.054289 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2926-96cc-492a-a9a4-b50754c6ce4f admin admin] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:39.054572 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2926-96cc-492a-a9a4-b50754c6ce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:39.054846 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2926-96cc-492a-a9a4-b50754c6ce4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:39.055519 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a2926-96cc-492a-a9a4-b50754c6ce4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:39.057141 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a2926-96cc-492a-a9a4-b50754c6ce4f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:39.067530 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1457/2907] 10.222.0.52 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:39.071488 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d831479-a6fa-4d47-b8c9-b86e161cc69c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:39.081723 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d831479-a6fa-4d47-b8c9-b86e161cc69c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:39.086252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:39.086882 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:39.087193 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:39.087462 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] PATH_INFO: `/v3/projects/6f960d6157f4416b857f85cf38d89455/users/c8fb862abfd346ae84876137ba42c7b9/roles/aa4ffe964b624a329cecff46b0a2c131` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:39.103389 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f960d6157f4416b857f85cf38d89455, user_id=c8fb862abfd346ae84876137ba42c7b9, role_id=aa4ffe964b624a329cecff46b0a2c131)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:39.106017 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:39.118289 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1451/2908] 10.222.0.52 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:39 2022] PUT /identity/v3/projects/6f960d6157f4416b857f85cf38d89455/users/c8fb862abfd346ae84876137ba42c7b9/roles/aa4ffe964b624a329cecff46b0a2c131 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:39.129878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a88cb0-1f22-4d43-8b80-0d7a9e0cc037 None None] REQUEST_METHOD: `POST` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:39.130164 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a88cb0-1f22-4d43-8b80-0d7a9e0cc037 None None] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:39.130435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a88cb0-1f22-4d43-8b80-0d7a9e0cc037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:39.155168 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:39.158146 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.auth.core [None req-a1a88cb0-1f22-4d43-8b80-0d7a9e0cc037 None None] MFA Rules not processed for user `c8fb862abfd346ae84876137ba42c7b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81457) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:39.160578 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a1a88cb0-1f22-4d43-8b80-0d7a9e0cc037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:39.223219 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1458/2909] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:09:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:39.234291 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:39.236070 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:39.292533 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7086535-5c10-43d9-a511-1f1137c9cb96 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:39.296373 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815b154-b47a-46d4-90db-266991af1c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75aa23cd162e4da2bd6a936c27e51f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:39.296970 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815b154-b47a-46d4-90db-266991af1c5a service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:39.297255 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815b154-b47a-46d4-90db-266991af1c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:39.297525 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815b154-b47a-46d4-90db-266991af1c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:39.305832 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815b154-b47a-46d4-90db-266991af1c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:39.307283 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815b154-b47a-46d4-90db-266991af1c5a service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:39.315954 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1452/2910] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:49.219830 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=26 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:49.311971 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:59.416645 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a88cb0-1f22-4d43-8b80-0d7a9e0cc037 None None] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:59.418016 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-a1a88cb0-1f22-4d43-8b80-0d7a9e0cc037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:59.480607 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a88cb0-1f22-4d43-8b80-0d7a9e0cc037 None None] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:59.484073 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:59.484666 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:59.484942 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:59.485207 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:59.486921 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:59.496188 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:59.497584 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:59.505423 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1459/2911] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:02.861848 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815b154-b47a-46d4-90db-266991af1c5a service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:02.871922 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815b154-b47a-46d4-90db-266991af1c5a service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:02.875540 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:02.876136 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:02.876418 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:02.876688 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:02.878476 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:02.888641 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:02.890055 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:02.898370 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1453/2912] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:03.009197 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.010668 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:03.025978 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e11f82-09a8-4f20-ab80-19c1ca8cefd0 service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.029871 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.030709 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.031655 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.032031 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.033885 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:03.106981 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.108407 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.119199 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1460/2913] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:03.182968 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.184344 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:03.252660 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f10cc-7f22-4ad5-aa2c-2e9009998949 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.257527 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70273225-1741-413e-8c50-c62067e3bc12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.258123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70273225-1741-413e-8c50-c62067e3bc12 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.258400 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70273225-1741-413e-8c50-c62067e3bc12 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.258669 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70273225-1741-413e-8c50-c62067e3bc12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.260392 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-70273225-1741-413e-8c50-c62067e3bc12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:03.273873 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70273225-1741-413e-8c50-c62067e3bc12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.275636 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70273225-1741-413e-8c50-c62067e3bc12 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.286415 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1454/2914] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:13.114297 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=60 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:13.281770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=38 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:14.317554 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:14.377581 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2df6f-ec65-4954-a6fb-de8bdd300b0d service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:14.382762 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:14.383414 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:14.383697 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:14.383969 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] PATH_INFO: `/v3/users/c8fb862abfd346ae84876137ba42c7b9` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:14.385092 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fb862abfd346ae84876137ba42c7b9)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:14.387032 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:14.435222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fb862abfd346ae84876137ba42c7b9'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:14.442069 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fb862abfd346ae84876137ba42c7b9'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:14.442730 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fb862abfd346ae84876137ba42c7b9'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:14.443378 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fb862abfd346ae84876137ba42c7b9'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:14.450777 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fb862abfd346ae84876137ba42c7b9'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:14.468767 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1461/2915] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:10:14 2022] DELETE /identity/v3/users/c8fb862abfd346ae84876137ba42c7b9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:14.473315 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f216e73-fe9b-46eb-a1d2-e80081599a8c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:14.473731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f216e73-fe9b-46eb-a1d2-e80081599a8c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:14.473852 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f216e73-fe9b-46eb-a1d2-e80081599a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:14.492827 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0f216e73-fe9b-46eb-a1d2-e80081599a8c None None] MFA Rules not processed for user `10d738bcb5484a248894a50e95518057`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:14.494023 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f216e73-fe9b-46eb-a1d2-e80081599a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:14.557519 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1455/2916] 10.222.0.52 () {58 vars in 958 bytes} [Mon Oct 10 12:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:14.563974 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:14.565412 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:14.623913 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54ae2a-61ea-43e3-9cd3-90d5ec0dc080 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:14.627589 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ad70aa047d443b97e13447397a8a15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:14.628227 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:14.628510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:14.628781 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:14.638352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:14.639852 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:14.650233 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1462/2917] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:14.917249 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f216e73-fe9b-46eb-a1d2-e80081599a8c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:14.918670 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f216e73-fe9b-46eb-a1d2-e80081599a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:14.931684 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f216e73-fe9b-46eb-a1d2-e80081599a8c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:14.936375 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:14.937009 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:14.937288 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:14.937567 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] PATH_INFO: `/v3/projects/6f960d6157f4416b857f85cf38d89455` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:14.938705 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f960d6157f4416b857f85cf38d89455)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:14.940648 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:14.988571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f960d6157f4416b857f85cf38d89455', 'request_id': 'req-91469359-e12d-4cac-ba9a-f3301f954383'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:14.998294 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f960d6157f4416b857f85cf38d89455', 'request_id': 'req-91469359-e12d-4cac-ba9a-f3301f954383'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:15.004950 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f960d6157f4416b857f85cf38d89455', 'request_id': 'req-91469359-e12d-4cac-ba9a-f3301f954383'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:15.016178 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] The token cache is being invalidate because project 6f960d6157f4416b857f85cf38d89455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:10:15.016463 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:15.017123 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:15.018598 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1456/2918] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:10:14 2022] DELETE /identity/v3/projects/6f960d6157f4416b857f85cf38d89455 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:10:15.580435 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:15.581994 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:15.610538 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:15.664042 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0029387-66c6-452e-9fa5-819e9257e14c service neutron] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:15.726753 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:15.727380 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:15.727656 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:15.727922 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:15.729639 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:15.741082 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:15.742628 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:15.752878 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1463/2919] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:25.012967 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:25.748569 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=52 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:11:27.678885 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:11:27.680504 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:27.703212 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91469359-e12d-4cac-ba9a-f3301f954383 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:11:27.709558 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:11:27.710252 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:27.710609 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:27.711179 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:27.713588 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:27.726979 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:11:27.729372 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:11:27.741506 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1457/2920] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:11:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:11:30.143550 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:11:30.155213 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f4de6-8760-4405-882d-92ec08cdc246 service nova] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:11:30.161516 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152edb19fd7b434485b0e1ebc5078335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:11:30.162806 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] REQUEST_METHOD: `GET` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:30.163190 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:30.163548 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:30.165322 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:30.178693 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:11:30.180303 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:11:30.191934 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1464/2921] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:11:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:11:37.429287 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:11:37.431505 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:37.501498 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77463379-196b-4fe3-a578-883fe483eb86 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:11:37.506022 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:11:37.506642 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:37.506957 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:37.507235 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] PATH_INFO: `/v3/users/fe67ba70aef540e98dc56fe609077f93` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:37.508347 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe67ba70aef540e98dc56fe609077f93)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:11:37.510821 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:11:37.556473 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe67ba70aef540e98dc56fe609077f93'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.563390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe67ba70aef540e98dc56fe609077f93'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.564098 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe67ba70aef540e98dc56fe609077f93'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.564731 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe67ba70aef540e98dc56fe609077f93'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.572059 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe67ba70aef540e98dc56fe609077f93'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.589687 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1458/2922] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:11:37 2022] DELETE /identity/v3/users/fe67ba70aef540e98dc56fe609077f93 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:11:37.774284 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:11:37.785745 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f7af9-fd6e-4609-8017-969c28ed5842 service nova] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:11:37.842133 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:11:37.842769 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:37.843080 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:37.843358 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] PATH_INFO: `/v3/projects/52575749f6f6409b947c28bb16649eb7` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:37.844470 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52575749f6f6409b947c28bb16649eb7)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:11:37.846375 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:11:37.891901 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52575749f6f6409b947c28bb16649eb7', 'request_id': 'req-1cdc1693-2c58-4e33-82dd-85fd868c8464'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.898927 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52575749f6f6409b947c28bb16649eb7', 'request_id': 'req-1cdc1693-2c58-4e33-82dd-85fd868c8464'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.905785 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52575749f6f6409b947c28bb16649eb7', 'request_id': 'req-1cdc1693-2c58-4e33-82dd-85fd868c8464'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.916222 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] The token cache is being invalidate because project 52575749f6f6409b947c28bb16649eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:11:37.916510 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.917135 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:37.918592 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1465/2923] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:11:37 2022] DELETE /identity/v3/projects/52575749f6f6409b947c28bb16649eb7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:11:37.922990 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:11:37.924390 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:37.937317 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31de34-498f-4bd3-bed3-3759cc42772c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:11:37.994627 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:11:37.995307 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:37.995571 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:37.995824 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] PATH_INFO: `/v3/users/f55d7e6e42a14247bbe2ae926eecf8c9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:37.997203 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55d7e6e42a14247bbe2ae926eecf8c9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:11:37.999454 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:11:38.004765 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:11:38.045311 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55d7e6e42a14247bbe2ae926eecf8c9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.052429 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55d7e6e42a14247bbe2ae926eecf8c9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.053080 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55d7e6e42a14247bbe2ae926eecf8c9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.053675 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55d7e6e42a14247bbe2ae926eecf8c9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.061451 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55d7e6e42a14247bbe2ae926eecf8c9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.079391 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1459/2924] 10.222.0.52 () {58 vars in 1224 bytes} [Mon Oct 10 12:11:37 2022] DELETE /identity/v3/users/f55d7e6e42a14247bbe2ae926eecf8c9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:11:38.244118 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] Authenticating user token {{(pid=81457) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:11:38.245928 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.fernet_utils [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81457) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:38.259352 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc1693-2c58-4e33-82dd-85fd868c8464 admin admin] Validating token access rules against request {{(pid=81457) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:11:38.318083 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d738bcb5484a248894a50e95518057', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '992fcf5a85d6497db94cb36327aec1f0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81457) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:11:38.318720 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:38.319027 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] SCRIPT_NAME: `/identity` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:38.319298 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] PATH_INFO: `/v3/projects/efedebc0f9044fa9bed771719507bd1d` {{(pid=81457) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:38.320474 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] RBAC: Authorizing `identity:delete_project(project_id=efedebc0f9044fa9bed771719507bd1d)` {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:11:38.322446 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] RBAC: Authorization granted {{(pid=81457) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:11:38.344242 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:11:38.378542 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efedebc0f9044fa9bed771719507bd1d', 'request_id': 'req-d80d8388-fc2d-43a3-8325-c1ea42dc454f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.385987 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efedebc0f9044fa9bed771719507bd1d', 'request_id': 'req-d80d8388-fc2d-43a3-8325-c1ea42dc454f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.392842 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efedebc0f9044fa9bed771719507bd1d', 'request_id': 'req-d80d8388-fc2d-43a3-8325-c1ea42dc454f'} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.408523 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] The token cache is being invalidate because project efedebc0f9044fa9bed771719507bd1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81457) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:11:38.408793 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.409437 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG keystone.notifications [None req-d80d8388-fc2d-43a3-8325-c1ea42dc454f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81457) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:38.410952 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: [pid: 81457|app: 0|req: 1466/2925] 10.222.0.52 () {58 vars in 1233 bytes} [Mon Oct 10 12:11:38 2022] DELETE /identity/v3/projects/efedebc0f9044fa9bed771719507bd1d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:11:48.075770 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:11:48.401580 dv-cinder-850618-4-iscsi devstack@keystone.service[81457]: DEBUG dbcounter [-] [81457] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81457) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:23.773202 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:23.774646 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:23.843069 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4b2b9-c1e9-4217-91a8-14226f950364 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:23.900794 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7eea4-7cd9-46e1-9dbc-2abfc59908ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e6c2f60d80647888998040638a5b993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fad1c962c6024e5cbb001d3256203928', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:23.901396 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7eea4-7cd9-46e1-9dbc-2abfc59908ef service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:23.901674 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7eea4-7cd9-46e1-9dbc-2abfc59908ef service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:23.901944 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7eea4-7cd9-46e1-9dbc-2abfc59908ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:23.903679 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10c7eea4-7cd9-46e1-9dbc-2abfc59908ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:23.914579 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7eea4-7cd9-46e1-9dbc-2abfc59908ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:23.916188 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7eea4-7cd9-46e1-9dbc-2abfc59908ef service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:23.926243 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1460/2926] 10.222.0.52 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:33.921681 dv-cinder-850618-4-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=58 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}